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Abstract 

An  initiative  to  expand  the  use  of  parametric  estimating  in  government  contracting 
has  enjoyed  only  marginal  success.  Some  reasons  for  the  slower  pace  of  implementation 
are  the  lack  of  availability  of  historical  data,  training,  acquisition  rules,  utility,  and 
accuracy  concerns.  Those  associated  with  the  Parametric  Cost  Estimating  Initiative 
(PCEI)  have  a  higher  perception  of  parametric  utility  and  accuracy  than  those  not 
associated  with  the  initiative.  The  PCEI  advocates  using  parametric  techniques  in  any 
situation,  not  just  the  concept/development  phases  advocated  by  most  literature. 

Parametric  estimating  is  a  catch-all  term  for  several  different  types  of  statistically 
based  methodologies,  ranging  from  simple  CERs  to  complex  models.  Each  type  has 
different  implementation  requirements.  Using  the  single  term  parametrics  can  be  very 
deceiving,  making  it  difficult  for  management  to  allocate  resources  necessary  for 
implemetation. 

PCEI  and  non-PCEI  acquisition  groups  have  similar  perceptions  of  accuracy  and 
utility  for  CERs,  but  different  perceptions  for  more  complex  parametric  models.  The 
PCEI  has  been  successful  expanding  the  use  of  parametrics,  though  not  significantly 
enough  to  be  persuasive.  Most  of  the  PCEI  successes  have  been  related  to  CERs,  not 
complex  models.  Advances  in  complex  model  use  has  occurred  with  contractors  who 
routinely  develop  state  of  the  art  hardware. 
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AN  EXPLORATORY  STUDY  OF  THE  USE  OF 
PARAMETRIC  ESTIMATING 
IN  DEPARTMENT  OF  DEFENSE  CONTRACTING 


I.  Introduction 


We  are  in  the  midst  of  change.  Certainly  the  nature  of  acquisition  has  changed, 
affected  by  smaller  budgets,  smaller  forces,  and  a  focus  on  efficiency  in  all  aspects  of 
commercial  and  government  business  operations.  Every  phase  of  the  Government's 
acquisition  process  is  being  reviewed  and  reengineered  to  improve  its  efficiency 
(Statutory/Policy  Changes,  1998:1).  In  the  past,  the  acquisition  of  new  weapon  systems 
was  characterized  by  actual  costs  ultimately  being  greater  than  their  original  estimated 
costs.  The  1990's  has  seen  acquisition  reform  and  affordability  emphasized  as  primary 
goals  within  the  Department  of  Defense  (DoD),  with  commercial  practices  being 
encouraged  and  non-value  added  tasks  being  discouraged  (Widmann  and  others, 
1997:433). 

One  area  of  acquisition  reform  being  examined  is  improvement  of  cost  estimating 
tools  and  processes.  The  emphasis  on  downsizing  and  budget  cuts  will  no  longer  support 
ineffective  cost  estimating.  Our  study  investigates  whether  current  acquisition  cost 
estimating  policies  add  value  and  are  perceived  to  be  effective  in  predicting  future  DoD 
weapon  system  cost.  Current  estimating  practices  are  considered  inefficient  because  the 


processes  are  labor  intensive,  cycle  times  are  long,  and  the  tools  and  processes  have  not 
always  provided  the  best  and  most  timely  results  (Spector,  1995:1).  The  environment 
requires  the  cost  estimating  community  consider  different  tools  and  processes  which  may 
provide  more  credible  estimates  and,  at  the  same,  time  reduce  the  substantial  labor 
involvement.  The  time  is  ripe  for  the  introduction  of  new  computer-driven  techniques  if 
those  techniques  can  provide  accuracy  and  time  savings.  The  need  to  be  more  efficient 
and  reduce  costs  in  the  DoD  has  led  to  the  Parametric  Cost  Estimating  Initiative  (PCEI). 

Cost  Estimating  Methodologies 

Cost  estimating  has  played  a  fundamental  role  in  the  acquisition  process  (Zhang  and 
others,  1996:95;  McDonald,  1992:D.3.1;  Smith  and  Mason,  1997:137).  It  provides 
information  to  make  trade-off  decisions  between  alternative  solutions  and  establishes  the 
standards  of  acceptability  against  which  the  effectiveness  of  the  solutions  are  measured 
(Zhang  and  others,  1996:95;  Doraiswamy  and  Hunt,  1994:Q.M.4.4).  Cost  estimating  is 
essentially  a  computational  process  used  to  predict  the  future  cost  of  a  system  or  product. 
Very  rarely  are  all  the  parameters  or  conditions  known  when  the  estimate  is  prepared. 

There  are  three  basic  estimating  methodologies:  cost  build-up  or  bottoms-up, 
analogy,  and  parametrics  (Widmann  and  others,  1997:434).  A  detailed  discussion  of  the 
methodologies  is  provided  in  Chapter  II.  In  general,  the  methods  differ  based  on  the 
information  available,  the  nature  of  the  project  or  program,  and  the  time  available  to 
prepare  the  estimate  (De  la  Garza  and  Rouohana,  1995:14).  The  cost  build-up  method 
uses  the  most  currently  available  information  or  expert  judgment  obtained  from  managers 
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responsible  for  performing  the  task  and  derives  a  total  cost  by  summation  of  the  various 
tasks  and  cost  elements  (Widmann  and  others,  1 997 :434).  The  analogy  method  typically 
finds  historical  data  for  previously  built  similar  items  or  tasks,  and  applies  a  subjective 
adjustment  factor  based  on  the  degree  of  increased  or  decreased  difficulty  of  producing 
the  particular  item  or  performing  the  particular  task.  The  cost  build-up  and  analogy 
methods  of  estimating  are  commonly  used  in  contracting  and  are  generally  well 
understood.  However,  obtaining  the  information  to  prepare  these  types  of  cost  estimates 
is  often  difficult  and  time  consuming,  especially  early  in  an  acquisition  cycle.  The  third 
basic  method  is  parametrics.  Parametric  applications  imply  the  use  of  estimating 
relationships  or  models  to  predict  elements  of  cost  or  total  cost. 

Parametric  Estimating  Applications 

Once  it  has  been  determined  to  use  parametric  estimating,  there  are  choices  about 
which  type  of  parametric  application  to  use.  Parametric  estimating  is  an  encompassing 
term  which  includes  everything  from  a  simple  cost  estimating  relationship  (CER)  that 
estimates  one  element  of  cost,  to  complex  proprietary  commercial  models  that  estimate 
the  entire  cost  of  a  procurement.  CERs  are  mathematical  expressions  that  relate  cost  as 
the  dependent  variable  to  one  or  more  independent  cost-driving  variables  (DoD, 

1995:35).  There  are  two  types  of  CERs  -  cost-to-cost  and  cost-to-non-cost.  Estimating 
quality  hours  using  manufacturing  hours  as  a  base  is  an  example  of  a  cost-to-cost  CER. 
Development  of  overhead  costs  also  involves  the  application  of  cost-to-cost  CERs  since  it 
relates  overhead  costs  to  some  direct  base.  Estimating  technical  publication  costs  using  a 
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dollar  per  page  factor  is  an  example  of  a  cost-to-non-cost  CER.  Another  example  is  the 
use  of  the  dollars  per  pound  CER  to  approximate  airframe  cost.  When  numerous  CERs 
are  used  in  combination  to  derive  an  estimate,  the  combination  of  CERs  becomes  a 
parametric  model.  Parametric  models  can  be  used  to  estimate  a  significant  portion  or  the 
entire  cost  of  a  proposal  (DoD,  1995:9). 

Parametric  applications  vary  significantly  in  how  difficult  they  are  to  develop,  use, 
explain,  and  how  much  they  are  currently  used.  Cost-to-cost  CERs  are  very  popular  and 
simple  to  develop,  use,  and  explain.  Commercial  models  that  develop  entire  proposal 
costs  are  very  complex,  require  specialized  training,  and  have  been  used  sparingly  in 
establishing  firm  contract  prices. 

Research  Objective 

Since  the  formation  of  the  International  Society  of  Parametric  Analyst  (ISP A)  in 
1978,  there  have  been  numerous  efforts  to  promote  the  use  of  parametric  estimating  in 
commercial  and  government  contracting.  The  belief  has  long  existed  that  it  takes  too 
long  to  prepare,  evaluate,  and  negotiate  estimates,  and  that  parametrics  can  achieve  the 
same  level  of  accuracy  more  efficiently.  The  PCEI  provides  the  current  push  to  expand 
the  use  of  parametric  estimating  into  the  contracting  process. 

The  PCEI  is  attempting  to  expand  the  use  of  parametric  methods  over  other  basic 
estimating  methods  in  situations  where  it  has  not  been  the  preferred  method.  This 
expanded  use  of  the  parametric  method  involves  all  types  of  parametric  applications.  The 
objective  of  this  research  is  to  provide  a  comparison  between  the  established 
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understanding  of  parametric  uses,  and  compare  that  baseline  understanding  to  the 
positions  advocated  by  the  PCEI.  In  doing  so,  the  research  will  evaluate  some  of  the 
fundamental  positions  of  the  PCEI.  To  be  successful,  the  PCEI  needs  to  know  what 
situations  parametrics  would  be  most  useful  and  when  used,  which  application  would  be 
most  useful. 

Another  objective  of  this  research  is  to  examine  why  parametrics  have  not  been 
more  widely  used  in  DoD  firm  contracting  proposals.  An  important  aspect  of  this  is  DoD 
customer  perceptions  of  parametric  estimating  and  how  those  perceptions  compare  to  the 
baseline  understanding  of  parametrics  and  the  positions  advocated  by  the  PCEI. 

The  PCEI's  plan  expands  the  use  of  parametric  estimating  into  the  area  of 
establishing  firm  contract  prices.  This  has  not  happened  in  the  past  because  the  cost 
build-up  and  analogy  approaches  were  believed  to  be  most  accurate.  However, 
significant  effort  to  expand  the  use  of  parametric  estimating  into  DoD  contracting 
continues  because  current  processes  are  considered  inefficient.  It  is  also  believed  that 
parametric  estimating  is  a  better  way  of  preparing  an  estimate,  combining  accuracy  and 
efficiency.  These  and  related  assertions  are  causing  the  expenditure  of  extensive 
resources  to  expand  the  role  of  parametric  estimating  in  the  contracting  arena.  If 
parametric  estimating  is  inappropriate  for  the  purposes  planned,  then  the  resources 
currently  being  devoted  to  implementing  it  are  being  wasted.  The  improper  use  of 
parametric  estimating  could  lead  to  inaccurate  estimates  causing  additional  wasted 
resources,  improperly  funded  programs,  and  associated  acquisition  perturbations. 

Without  this  investigation  into  perceived  differences  between  cost  estimating  techniques, 
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practitioners  and  researchers  may  be  basing  important  decisions  on  inadequate 
information. 

The  current  environment  focuses  on  getting  quicker  and  cheaper  products  and 
services  but  still  emphasizes  processes  that  make  sense  and  produce  the  best  results. 
Where  can  real  efficiencies  be  achieved  and  top  quality  products  obtained?  To  make  a 
determination  about  the  proper  uses  of  parametric  estimating,  the  principles  and 
techniques  germane  to  the  field  of  cost  estimating  must  be  reviewed  and  understood. 
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II.  Literature  Review 


Cost  estimating  is  the  process  of  forecasting  a  future  result  in  terms  of  cost,  based 
upon  currently  available  information  such  as  historical  data  or  observations  (ASPM, 
1975:A1-B6;  ASD,  1989:1 10).  Estimates  are  derived  for  many  purposes  and  during 
several  stages  of  the  acquisition  process.  Many  considerations  go  into  deriving  these  cost 
estimates.  While  the  considerations  to  determine  the  appropriate  method  are  intertwined, 
the  overriding  considerations  are:  purpose  of  the  intended  estimate;  quality  of  the 
information  available  to  develop  the  estimate;  and  the  time  and  resource  constraints 
devoted  to  developing  the  estimate.  Examining  the  traditional  intended  purpose  and  use 
of  parametric  estimating  requires  a  review  and  understanding  of  the  general  concept  of 
cost  estimation.  To  properly  address  this,  we  review  the  general  field  of  cost  estimation 
and  how  that  general  body  of  knowledge  influences  the  use  of  parametric  estimating. 
Following  the  cost  estimation  area  will  be  a  review  of  the  Parametric  Cost  Estimating 
Initiative  (PCEI).  The  establishment  of  the  initiative  will  be  discussed,  along  with  the 
PCEI's  goals  and  objectives.  A  chapter  summary  closes  the  chapter. 

Cost  Estimating:  Establishing  a  Baseline  for  Parametric  Estimating 

This  section  of  the  literature  review  will  provide  a  theoretical  perspective  of  the 
types  and  purposes  of  estimating  methods.  In  addition,  it  will  provide  a  working 
definition  of  paramedics,  examine  accuracy  and  risk  analysis,  and  present  the  potential 
usefulness  or  utility  of  parametric  models  as  contained  in  the  literature. 
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Estimating  Methods,  Definition  and  Purposes.  The  literature  supports  classifying 
cost  estimating  techniques  by  both  the  method  employed  to  prepare  a  cost  estimate  and 
the  intended  purpose  of  the  estimate.  Both  the  method  and  purpose  classification  systems 
produce  similar  results.  A  discussion  of  the  cost  estimating  methods  will  be  considered 
first,  followed  by  an  amplification  of  the  parametric  method  and  its  characteristics  and 
definitions. 

There  are  three  primary  methods  of  cost  estimation  techniques  used  to  prepare 
estimates:  1)  cost  build-up  or  bottoms-up,  2)  analogy,  and  3)  parametrics  (Widmann  and 
others,  1997:434).  The  cost  build-up  methodology  is  perceived  as  being  more  certain  or 
accurate  and  the  parametric  methodology  as  being  least  certain.  The  cost  build-up 
approach  is  more  certain  because  it  provides  a  total  product  cost  based  on  a  summation  of 
individual  task  and  cost  elements.  Normally  used  during  the  mature  portions  of  the 
acquisition  cycle,  cost  build-up  produces  fairly  accurate  estimates  based  on  information 
about  the  detailed  design  of  the  product  or  system  (Widmann  and  others,  1997:434). 
Actual  or  quoted  cost  information  is  available  on  material  costs  and  processes.  Labor 
costs,  as  a  further  example,  can  be  accurately  determined  based  on  the  expert  judgment 
from  the  engineers  or  specialists  who  perform  the  task.  The  final  cost  is  determined  by 
summarizing  the  individual  cost  elements  and  applying  overhead  rates  or  other  estimating 
relationships. 

The  analogy  method  is  used  when  there  is  a  sufficient  degree  of  similarity  between 
the  product  being  developed  and  an  existing  product  (Widmann  and  others,  1997:434). 
Historical  data  from  the  existing  product  is  used  to  provide  a  basic  cost  value  for  the  new 
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product.  An  adjustment  factor  is  then  applied  based  on  the  degree  of  increased  or 
decreased  difficulty  in  producing  the  product  or  performing  the  task. 

Both  the  cost  build-up  and  analogy  methods  of  estimating  are  commonly  used  in 
DoD  contracting  and  are  generally  well  understood.  The  third  method,  parametrics,  is  a 
paradox.  Certain  types  of  parametrics  are  simple  and  well  understood,  others  complex. 

Definitions  and  Categories  of  Parametric  Estimating.  The  literature  contains 
numerous  and  varied  definitions  for  the  broad  term  of  parametric  estimating.  There  are 
many  types  of  parametric  estimating  techniques,  some  very  simple,  well  understood  and 
commonly  used,  and  some  very  complex.  An  analysis  of  the  various  definitions 
contained  in  literature  provides  a  means  of  categorizing  the  types  of  parametric 
estimating  applications  according  to  their  use  and  complexity.  As  noted  above,  the  scope 
of  parametric  estimating  is  very  large  and  many  applications  of  the  technique  are  used  for 
various  purposes.  Simple  parametric  applications,  like  estimating  relationships  at  the  cost 
element  level,  are  used  in  creating  portions  of  firm  prices.  Conceptual  estimates  use  more 
complex  parametric  models  which  may  develop  all  or  part  of  the  estimate. 

Most  of  the  literature  characterizes  parametrics  in  terms  of  mathematical 
relationships.  Some  theorists  and  practitioners  treat  it  as  a  special  branch  of  econometric 
analysis  involving  cost  functions,  micro  constructs,  model  validation  procedures,  and 
forecasting  using  a  full  range  of  statistical  tools  (Parametric  Cost  Estimating  Course 
Manual,  1987:4).  Cost  Estimating  Relationships  (CERs)  are  a  common  component  in 
many  definitions.  To  that  end,  parametrics  has  been  described  as  an  estimate  that  uses 
one  or  more  CERs  and  associated  mathematical  algorithms  or  logic  to  establish  cost 
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estimates  (DoD,  1995:165).  CERs  are  used  to  measure  and  estimate  the  cost  associated 
with  the  development,  manufacture,  or  modification  of  a  specified  end  item,  basing  the 
measurement  on  technical,  physical  or  other  end  item  characteristics  (DoD,  1995:198). 

CERs  can  be  based  on  either  cost-to-cost  or  cost-to-non-cost  type  relationships.  The 
difference  between  the  two  types  of  CERs  is  the  base  of  the  estimating  relationship.  If 
the  base  is  a  cost  related  parameter,  the  CER  is  cost-to-cost.  An  example  is  the 
commonly  used  approach  to  estimate  overhead  where  the  base  is  a  cost  item  like 
manufacturing  or  material  dollars.  If  the  base  of  the  CER  is  a  physical  attribute 
associated  with  the  end  item,  the  CER  is  cost-to-non-cost.  Cost-to-non-cost  CERs 
include  hours  per  drawing  or  dollars  per  pound  relationships.  In  other  words,  CERs  are 
mathematical  expressions  which  relate  the  dependent  variable  of  cost  to  the  independent 
variables  (DoD,  1995:165). 

More  complex  parametrics  applications  can  be  characterized  in  terms  of  the  types  of 
models  used.  These  models  may  be  simple  cost  estimating  relationships  or  they  may  be 
quite  complex  involving  linked  relationships  (Parametric  Cost  Estimating  Course 
Manual,  1987:4).  The  term  linkage  implies  that  a  change  in  an  item's  design  affects  the 
related  cost  of  manufacturing  and  other  support  functions.  A  parametric  model  accounts 
for  these  related  and  cost  impacts.  A  parametric  model  is  a  group  of  CERs  used  together 
to  estimate  the  entire  proposal  or  a  significant  portion  of  the  proposal  (DoD,  1995:9). 

There  are  two  main  types  of  cost  models:  in-house  and  commercially  available 
models.  In-house  models  are  end  item  contractor  developed  models.  The  historical  data 
used  to  develop  the  model  is  almost  exclusively  from  the  end-item  contractor's 
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accounting  and  management  system.  Commercially  available  models  are  developed  by 
companies  whose  sole  business  is  developing  models  to  be  used  by  virtually  any 
contractor  in  any  business  or  any  product.  The  data  used  to  develop  these  models  is 
obtained  from  numerous  contractors  and  programs  over  a  long  period  of  time. 
Commercial  models  have  to  be  adapted  to  the  end  item  contractor  through  a  calibration 
and  validation  process.  Calibration  and  validation  are  procedures  for  adjusting  a 
commercial  model  to  a  specific  contractor  because  of  differences  in  his  processes, 
efficiencies  or  any  other  contractor  specific  peculiarities  (DoD,  1995:95).  Both  types  of 
models  can  estimate  portions  of  or  the  entire  cost  of  a  proposal. 

From  these  definitions  and  characterizations,  several  key  terms  and  concepts  are 
presented.  Paramedics  include  cost  functions  and  forecasting  cost  estimates  using 
statistical  tools.  Paramedics  requires  CERs  based  on  cost,  technical,  physical,  and  other 
end  item  characteristics  and  models  that  will  include  CERs  that  will  estimate  portions  of 
or  the  entire  cost  of  a  product.  Many  definitions  incorporate  the  terms  above  in  various 
combinations. 

In  summary,  key  concepts  in  the  literature  about  the  types  of  CERs  and  cost  models 
indicate  six  categories  of  parametric  application  can  be  established:  1)  CERs  that  are 
cost-to-cost,  2)  CERs  that  are  cost-to-non-cost,  3)  commercial  models  that  estimate  the 
entire  cost  of  a  proposal,  4)  commercial  models  that  estimate  only  significant  portions  of 
a  proposal,  5)  in-house  models  that  estimate  the  entire  cost  of  a  proposal,  and  6)  in-house 
models  that  estimate  only  significant  portions  of  a  proposal. 
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Purpose  of  Estimate.  The  previous  sections  categorized  estimates  based  on  the 
methodology  used.  Types  of  estimates  can  also  be  categorized  based  on  the  intended 
purpose  of  the  estimate.  Remer  lists  the  three  purposes  as  definitive,  budgetary,  and 
order  of  magnitude,  in  descending  order  of  certainty  (Remer  and  Buchanan,  1993:8). 

A  definitive  cost  estimate  is  used  during  the  more  mature  portions  of  a  production 
program,  when  a  firm  design  exits.  The  estimate  is  based  on  the  firm  design  and 
schedules  that  are  clearly  defined  (Remer  and  Buchanan,  1993:9).  Actual  cost 
information  or  cost  history  is  available  to  create  the  estimate.  Therefore,  a  definitive  cost 
estimate  is  produced  using  the  cost  build-up  methodology  discussed  previously. 

Budgetary  cost  estimates  are  used  during  the  development  portion  of  a  program, 
where  there  is  an  overall  system  functional  requirement  with  at  least  preliminary 
deliverable,  receivable,  and  schedule  information  presented  for  the  requirement  (Remer 
and  Buchanan,  1993:9).  With  a  sufficient  degree  of  design  definition,  judgments  can  be 
made  about  similarities  between  the  product  to  be  developed  and  existing  products.  A 
budgetary  estimate  can  be  produced  using  the  analogy  method  of  cost  estimation. 

The  Order  of  Magnitude,  or  rough  order  of  magnitude  (ROM),  cost  estimate  is 
developed  early  in  a  program  during  the  requirements  definition  stage.  Only  preliminary 
information  about  the  requirement  and  the  deliverables  is  available  at  this  stage  (Remer 
and  Buchanan,  1993:9).  The  estimate  is  produced  using  a  minimal  set  of  inputs  to 
characterize  key  cost  drivers  using  a  parametric  method  of  cost  estimating.  The  literature 
suggests  that  paramedics  are  the  primary  means  of  developing  an  estimate  early  in  the 
program  (Black,  1987:1). 
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Accuracy.  Addressing  the  subject  of  relative  accuracy  between  parametric 
estimating  models  and  more  conventional  estimating  techniques  is  difficult.  An  objective 
direct  comparison  would  involve  evaluating,  over  time,  the  actual  final  costs  against  the 
estimated  costs  using  both  parametric  and  conventional  methodologies.  The  literature  is 
void  of  any  such  objective  comparisons,  probably  because  of  the  costs  associated  with 
preparing  and  administering  dual  estimates.  Estimation  performance  benefits  from 
learning  but  as  research  on  estimation  practices  and  accuracy  suggests,  there  is  little 
motivation  to  increase  learning  in  this  area  (Ogunlana,  1991 :133).  Estimation 
performance  is  considered  adequate  because  there  is  no  evidence  to  the  contrary.  This  is 
because  a  system  does  not  exist  which  would  require  regular  monitoring  of  estimation 
performance  (Ogunlana,  1991:133;  Lewis  and  Pearson,  1977:2).  A  structure  that  could 
determine  the  accuracy  of  estimates  and  identify  the  reasons  for  inaccuracies  would  be  an 
effective  aid  but  appears  difficult  to  institutionalize.  In  the  past,  workload  priorities, 
personnel  mobility,  and  accounting  systems  have  created  barriers  to  creating  such  a 
structure.  Without  a  structure  to  measure  the  accuracy  of  an  estimate,  there  cannot  be  a 
structure  to  objectively  compare  the  relative  accuracy  of  parametric  estimates  to 
conventional  techniques.  The  literature  therefore  discusses  relative  accuracy  between 
parametric  and  conventional  techniques  from  at  least  four  subjective  perspectives:  1)  the 
historical  accuracy  of  the  estimated  cost,  2)  the  maturity  of  the  effort  being  estimated,  3) 
the  resources  devoted  to  produce  an  estimate,  and  4)  the  techniques  used  to  provide  more 
accurate  estimates. 
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Historical  Accuracy.  The  literature  suggests  that  estimating  has  not  been  as 
accurate  as  desired.  In  the  1970's,  the  big  concern  was  cost  growth,  specifically  the  big 
difference  between  the  final  cost  and  the  initial  estimate  (Lewis  and  Pearson,  1977:2). 
The  General  Accounting  Office  (GAO)  estimates  67  percent  of  programs  overrun  by 
more  than  100  percent  based  on  the  unavailability  of  adequate  technical  information 
(Gardner  and  Passarello,  1981:2).  Strangely,  nothing  is  stated  about  the  overrun  in  terms 
of  the  nature  of  the  initial  estimate  and  how  it  was  estimated.  Another  study  of  computer 
managers  suggests  only  25  percent  of  the  projects  complete  close  to  their  estimates 
(Lederer,  1992b:86).  From  another  perspective,  Renter's  study  of  technically  advanced 
commercial  energy  and  chemical  programs  found  final  costs  exceed  initial  estimates  by 
an  average  of  104  percent  (Remer  and  Buchanan,  1993:7).  While  it  is  easy  to  see  that 
initial  estimates  are  not  as  accurate  as  desired,  it  is  difficult  to  determine  what  the 
estimates  referred  to  and  how  they  were  derived.  History  supports  the  need  for  more 
accurate  estimates.  The  following  perspectives  suggest  factors  that  affect  the  accuracy  of 
an  estimate. 

Maturity  of  Effort.  The  maturity  of  any  effort  has  been  shown  to  influence  the 
accuracy  of  the  estimate.  It  is  more  difficult  for  an  estimate  prepared  for  the  conceptual 
phase  of  a  program  or  an  estimate  prepared  for  a  preliminary  budget  to  compete  in 
accuracy  with  an  estimate  prepared  for  a  production  phase  or  an  estimate  prepared  for  a 
funds  authorization  requirement.  The  information  that  is  available  at  those  points  in  the 
acquisition  process  explains  the  difficulty  in  achieving  the  accuracy  (Lewis  and  Pearson, 
1977:2). 
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Research  has  shown  that  there  is  an  expected  level  of  accuracy  based  on  final  costs 
relative  to  the  initial  estimates  as  depicted  below  (Remer  and  Buchanan,  1993:8;  Chopey, 
1994:106): 

Type  of  Estimate 

and  Information  Available  Level  of  Accuracy 

Definitive,  Detailed  -  5%  to +15% 

Budgetary,  Preliminary  - 1 5%  to  +30% 

Order  of  Magnitude,  Conceptual  -30%  to  +50% 

Estimating  accuracy  improves  over  time  as  the  product  matures.  Information  used  to 

estimate  the  cost  of  an  item  never  before  designed  or  built  differs  greatly  from  the 

information  available  and  used  to  estimate  the  cost  after  producing  the  item.  During 

concept  exploration,  the  best  available  information  may  be  physical  and  performance 

characteristics  of  the  item.  During  production,  the  best  available  information  may  be  the 

previous  costs  of  production. 

Resources  Devoted.  The  effort  expended  to  produce  an  estimate  has  been 
shown  to  influence  the  accuracy  of  the  estimate.  The  Remer  study  is  often  cited  when 
reviewing  the  resources  expended  versus  accuracy  trade-off.  Remer  saw  that  frequent 
overruns  were  caused  by  the  lack  of  resources  spent  to  do  proper  up-front  estimating  and 
by  the  lack  of  implementor  involvement  in  the  estimating  process  (Remer  and  Buchanan, 
1993:9).  A  quantifiable  relationship  could  be  established  between  accuracy  and  the 
amount  of  resources  available  to  prepare  the  estimate,  defining  the  cost  of  preparing  an 
estimate  as  a  percentage  of  the  cost  of  the  item.  Increasing  levels  of  accuracy  cost  more 
resources,  regardless  of  business  type,  estimate  type,  or  dollar  value  (Remer  and 
Buchanan,  1993:26).  Results  from  Remer's  research  indicated  that  budget  estimates  cost 
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2.5  times  as  much  as  order  of  magnitude  estimates  and  definitive  estimates  cost  5  times 
as  much  as  order  of  magnitude  estimates  (Remer  and  Buchanan,  1993:9). 

A  related  study  by  Larsen  implies  a  relationship  between  profitability  and  estimating 
accuracy  (Larsen,  1994:25).  A  simulation  was  created  using  two  companies  making 
competitive  bids.  One  company  used  Remer's  guidelines  for  determining  the  amount  to 
invest  in  proposal  preparation  to  achieve  an  accuracy  level  for  a  definitive  estimate  and 
the  other  company  spent  far  less  on  its  estimate.  The  results  indicated  two  companies  of 
similar  size,  market  share,  and  cost  structure  will  have  substantially  different  average 
total  profits,  17  percent  versus  2.6  percent,  based  on  three  causes  (Larsen,  1994:26).  The 
accurate  estimating  company  will  seldom  have  negative  profits  and  will  also  earn  very 
high  profits  more  often  (Larsen,  1994:26).  The  inaccurate  estimating  company  will  have 
negative  profits  on  most  of  its  awarded  efforts  (Larsen,  1994:26).  The  point  of  Larsen 
and  Remer's  study  is  that  if  the  proper  amounts  of  resources  are  not  devoted  to  preparing 
an  estimate,  the  result  will  be  greater  inaccuracy  and  reduced  profitability.  Many 
parametric  models  are  used  to  make  rapid  ball-park  cost  estimates  of  both  recurring  and 
nonrecurring  manufacturing  cost  during  initial  design  phases  (Philpott  and  others, 
1996:256).  Parametrics  is  generally  perceived  to  be  more  effective  in  cost  and  time 
requirements  than  other  forms  of  estimating  (Hargrove,  1985:7).  While  cost  effective 
estimates  were  not  quantified  in  terms  of  accuracy  to  cost,  it  is  difficult  to  debate  the 
issue  that  parametric  estimates  are  quicker  and  cheaper  to  prepare.  When  the  quicker, 
cheaper  view  of  parametrics  is  combined  with  Remer  and  Larsen's  findings,  there  is  an 
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inference  that  quick  and  cheap  parametrics  are  less  accurate.  Parametric  estimates  may 
initially  cost  less  but  the  trade-off  may  be  less  accuracy. 

Techniques  to  Provide  More  Accurate  Estimates.  The  literature  provides 
guidelines  for  preparing  better  cost  estimates  which  do  not  necessarily  advocate 
parametric  estimating  as  more  accurate.  Among  them  are: 

-Assign  the  initial  estimating  tasks  to  the  final  developers. 

-Delay  finalizing  the  estimate  until  the  end  of  a  thorough  study. 

-Anticipate  and  control  changes. 

-Rely  on  documented  facts,  standards,  and  simple  arithmetic  formulas. 

-Do  not  rely  on  cost  estimating  software  for  an  accurate  estimate. 

(Lederer,  1992a:  57) 

Even  though  a  clear  comparison  cannot  be  made  between  the  accuracy  of  conventional 
estimating  versus  parametric  estimating,  the  literature  suggests  that  parametric  estimating 
may  not  be  as  accurate  as  other  methods.  The  historical  accuracy  of  initial  estimates  has 
not  been  very  good  when  compared  to  final  cost.  As  programs  mature  and  more 
information  is  available,  the  proper  use  of  that  information  leads  to  increased  estimating 
accuracy.  Conventional  methods  make  better  use  of  current  information  by  using  actual 
costs  of  previously  built  items  to  estimate  the  costs  of  the  next  essentially  identical  item. 
The  more  resources  put  into  an  estimate,  the  more  accurate  an  estimate  is  going  to  be 
(Remer  and  Buchanan,  1993:8).  Parametric  modeling  may  be  a  resource  saver,  but 
failure  to  use  the  more  current  available  information  to  prepare  an  estimate  will  likely 
cause  estimating  inaccuracies  and  the  associated  program  perturbations.  Conventional 
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estimating  methods  use  the  full  range  of  available  resources  to  develop  the  estimates. 
Certain  techniques  lead  to  more  accurate  estimates.  This  includes  waiting  as  long  as 
possible  to  make  the  estimate  and  getting  as  many  people  involved  as  possible.  This 
again  tends  to  favor  conventional  methods  over  parametric  modeling  in  terms  of 
accuracy. 

Risk  Analysis.  It  is  impossible  to  discuss  estimating  accuracy  without  discussing 
risk.  The  abundance  of  literature  in  this  area  reflects  the  importance  of  risk  analysis.  The 
1997  Journal  of  Cost  Analysis  contains  a  bibliography  on  the  topic  listing  fifty 
documents  on  the  subject  of  estimating  risk  and  how  to  develop  and  quantify  it  (Black, 
1997:61). 

Risk  exists  with  any  type  of  prepared  estimate  using  any  type  of  methodology.  Risk 
assessment  in  parametrics  is  important  because  of  the  black  box  nature  of  many 
parametric  model  predictions.  The  term  black  box  in  this  context  refers  to  providing  an 
input  to  a  model  and  getting  an  output  without  having  an  understanding  of  the 
mathematical  algorithms  that  produced  the  output.  Estimates  are  made  at  the  aggregate 
level,  not  on  a  task  by  task  basis  so  there  is  little  visibility  into  what  makes  up  the 
prediction.  Conventional  approaches,  however,  allow  intuitive  risk  assessments  to  be 
made  about  the  quality  and  accuracy  of  the  estimates.  Proper  implementation  of 
parametric  modeling  should  include  formal  risk  analysis. 

Historically,  cost  estimating  risk  has  contributed  less  to  the  cost  growth  of  weapon 
systems  than  schedule  or  technical  risk  (Biery  and  others,  1994:62).  However,  current 
research  suggests  that  estimating  uncertainties  do  account  for  about  25  percent  of 


18 


program  cost  growth  (Biery  and  others,  1994:62).  Sources  of  the  uncertainty  in 
parametric  estimating  techniques  include  the  imprecision  associated  with  the  specific 
technique  used,  and  input  errors  and  oversights  (Biery  and  others,  1994:60).  Another 
error  source  exists  with  the  historical  data  bases.  Uncertainties  are  created  both  with  the 
applicability  of  the  data  and  in  the  normalization  of  that  data. 

Industry  appears  to  understand  the  need  for  risk  analysis  as  shown  in  the  following 
statement.  "Parametric  cost  estimating  is  often  used  in  predicting  the  cost  of  something 
complex  that  has  never  been  built  before.  Risk  analysis  attempts  to  establish  a  range  for 
this  estimate  or  a  feeling  of  confidence  in  it"  (Black,  1987:1).  However,  government 
contracting  has  been  lacking  in  the  use  of  formal  risk  analysis  despite  a  1982  acquisition 
improvement  initiative  to  push  the  budget  to  most  likely  cost  and  apply  quantitative  risk 
techniques  (Biery  and  others,  1994:57). 

Utility  of  Parametric  Models.  The  next  sections  discuss  when  parametric  techniques 
should  be  used  and  how  useful  parametrics  methods  are  in  cost  estimation?  Based  on 
research  in  the  literature,  the  utility  of  parametric  estimating  focuses  in  the  following 
areas:  1)  intended  purpose  of  the  estimate,  2)  overall  customer  acceptance,  including 
training  and  comprehension,  and  3)  data  and  databases. 

Intended  Purpose.  Research  shows  that  the  intended  use  of  the  more  complex 
parametric  applications  is  during  the  early  program  phases  when  time  and  resource 
constraints  are  the  dictating  factors  (Philpott  and  others,  1996:257).  Parametrics  is 
"perceived  to  be  more  effective  in  cost  and  time  than  conventional  estimating" 
(Parametric  Cost  Estimating  Course  Manual,  1987:5).  Some  contractors  use  parametrics 
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estimates  to  test  the  estimates  that  were  prepared  using  conventional  techniques.  Others 
use  parametric  models  as  the  prime  source  of  the  estimate,  and  create  details  to  match  for 
submission  as  cost  or  pricing  data  to  the  government  (Hargrove,  1985:9). 

A  cost  build-up  proposal  states  every  task  that  is  going  to  be  accomplished  with  a 
value  attached  to  each  of  those  tasks.  As  a  result,  cost  build-up  proposals  become 
detailed  plans  for  accomplishment  of  a  project  or  modification  (Mann,  1997).  The  review 
of  a  cost  build-up  proposal  provides  insight  into  how  the  contractor  plans  to  accomplish  a 
task  and  the  importance  placed  on  certain  tasks.  Since  parametric  proposals  have  been 
generally  used  for  conceptual  and  planning  estimates,  the  absence  of  this  detailed  plan  is 
not  missed  (Mann,  1997). 

Customer  Acceptance.  Training  aids  in  the  understanding  or  comprehension  of 
parametric  models.  In  turn,  this  comprehension  is  necessary  for  customer  acceptance.  It 
is  important  to  understand  the  initial  effort  in  creating  a  parametric  estimate  in  order  to 
gain  acceptance  from  the  customer  (Stanley,  1981:1).  The  benefits  to  be  derived  from 
parametric  estimating  cannot  be  realized  until  the  models  are  understood  and  accepted  as 
a  reliable  prediction  of  cost  (Executive  Summary,  1 997a:  10).  Industry  has  noted  that  the 
"biggest  challenge  to  the  effective  implementation  of  parametrics  is  the  lack  of  familiarity 
with  parametric  estimating  concepts  on  the  part  of  the  program  and  government  oversight 
personnel"  (Lubell,  1997:8). 

Formal  training  is  one  aspect  in  achieving  a  successful  comprehension  level  of 
parametric  estimating  techniques  and  models.  However,  training  must  also  focus  on  four 
key  areas  to  ultimately  lead  to  customer  acceptance.  These  are:  1)  know  the  model,  2) 
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know  the  program,  3)  know  what  has  been  done,  and  4)  support  the  customer  (Stanley, 
1981:1). 

Basic  training  in  things  like  making  inputs  into  a  parametric  model  is  just  scratching 
the  surface  of  developing  a  good  parametric  estimator  (Philpott  and  others,  1996:256). 

The  estimator  must  understand  the  operation  of  the  model  to  be  able  to  sufficiently 
address  any  questions  about  the  model.  The  relationship  between  the  input  and  output 
data  and  how  to  analyze  that  data  must  be  clearly  understood  (Stanley,  1981 :5).  It  is 
necessary  to  focus  on  and  identify  any  key  cost  drivers  which  change  the  costs 
significantly  with  only  minor  variations  in  inputted  values  (Stanley,  1981:2).  It  is  also 
important  to  understand  the  relationship  between  product  design  parameters,  key  process 
parameters,  and  process  costs  (Philpott  and  others,  1996:257).  The  goal  is  that  enough 
comprehensible  data  is  provided  to  the  customer  so  that  the  customer  can  make  decisions 
(Stanley,  1981:5). 

Research  has  also  shown  that  designers  and  engineers  make  the  most  effective 
parametric  estimators  based  on  their  familiarity  with  the  pertinent  levels  of  technology 
and  technical  performance  (Apgar  and  Daschbach,  1987:9).  What  they  lack  in  parametric 
estimating  experience  is  quickly  replenished  by  the  state  of  the  art  in  estimating 
technology. 

To  achieve  customer  acceptance,  a  parametric  model  must  also  be  credible.  It  is  just 
as  important  to  consider  the  customer's  perception  of  the  model's  credibility  as  it  is  to 
consider  model  accuracy  and  efficiency  (Smith  and  Mason,  1997:156-157).  Management 
and  customer  confidence  in  parametric  methods  is  widely  recognized  as  a  problem 
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regardless  of  what  parametric  method  is  used  (Smith  and  Mason,  1997:156).  This  is 
particularly  true  in  firm  business  proposals,  which  must  satisfy  both  management  and  the 
customer  regarding  what  constitutes  a  proper  methodology.  In  the  cost  build-up 
estimation  technique,  there  is  a  credible  audit  trail  of  detailed  work  procedures  and 
methods,  materials,  and  schedules.  This  allows  any  assumptions  made  to  be  examined 
and  produces  a  perception  of  accuracy.  Parametric  methods  do  not  have  this  audit  trail. 
They  are  employed  because  it  is  either  not  feasible  or  not  cost  effective  to  develop  the 
detailed  level  of  cost  build-up.  Therefore,  in  complex  parametric  models,  the  detailed 
relationships  cannot  be  checked.  Acceptance  of  the  estimate  must  come  from  credibility 
of  the  model. 

Data  and  Databases.  Data  is  another  major  factor  influencing  the  usefulness  of 
parametric  estimating  techniques.  A  1974  Rand  Report  identified  the  greatest  limitation 
on  the  use  of  parametric  estimating  was  the  lack  of  cost  data  (Rose  and  High,  1974:6). 
Parametric  models  require  a  databank  of  credible  cost  information  to  include  actual  cost 
data  as  well  as  any  technical  or  specification  design  information.  The  data  must  be  in 
sufficient  detail  to  identify  the  factors  that  influence  cost  (Geaney,  1997:30).  By 
definition,  parametric  estimating  predicts  the  cost  of  a  new  product  by  extrapolating  from 
a  historical  database.  Therefore,  a  major  requirement  to  implement  parametric  estimating 
becomes  the  necessity  for  an  accurate  and  substantial  database  (Dean,  1997). 

One  challenge  in  building  a  database  is  the  need  for  more  than  just  historical  cost 
data  to  build  useful  cost  estimating  models.  A  correlation  of  the  historical  cost  data  to  the 
related  product  characteristics  is  needed  to  derive  the  cost  to  non-cost  relationships  and 
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derive  estimates  based  primarily  on  product  characteristics  (Parametric  Cost  Estimating 
Course  Manual,  1987:15).  There  is  some  question  whether  current  accounting  systems 
are  structured  to  provide  this  type  of  cost  data  to  product  correlation. 

Historical  data  can  also  be  difficult  to  interpret  because  contractors  organize  data 
differently  and  inconsistently  across  multiple  project  databases.  However,  certain 
guidelines  exist  which  can  lead  to  a  consistent  format  for  categorizing  historical  data. 

Data  from  Cost  Schedule  Control  Systems  Criteria  (C/SCSC)  and  the  MIL-STD-1567 
system  can  provide  formatted  sources  of  historical  data  for  parametric  estimating  (Mathis, 
undated:375).  Guidelines  that  establish  procedures  for  data  accumulation  and  exchange 
would  make  future  parametric  use  more  viable  because  the  data  used  for  the  parametric 
models  would  be  more  universally  accepted.  Data  is  one  of  the  main  concerns  of  the 
PCEI. 

The  Parametric  Cost  Estimating  Initiative 

Early  in  1994,  parametric  practitioners  met  to  evaluate  why  there  was  not  a  greater 
use  of  parametric  estimating  in  DoD  and  NASA  proposals;  to  identify  barriers  to  the 
expanded  use  of  parametrics;  and  to  take  advantage  of  any  identified  opportunities  (DoD, 
1995:8).  It  was  quickly  determined  no  formal  barriers  existed  in  implementing 
parametric  estimating  in  Government  contracting  (Hertling,  1998:9).  However,  there 
were  issues  that  needed  to  be  addressed  and  as  a  result  the  Parametric  Cost  Estimating 
Initiative  was  established. 
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The  PCEI  began  in  April  of  1994  with  the  formation  of  a  joint  govemment/industry 
working  group.  Action  items  established  by  the  working  group  were  to  expand  the  use  of 
parametric  estimating  techniques  for  government  proposals;  achieve  recognition  that 
parametrics  is  an  acceptable  method  of  cost  estimating;  and  establish  reinvention 
laboratories  to  test  the  expanded  use  of  parametrics  (Hertling,  1998:10).  The  PCEI  has 
been  successful  in  obtaining  industry  and  government  senior  management  support  and 
encouragement.  Both  the  Director  of  Defense  Procurement  and  the  Principle  Deputy 
Assistant  Secretary  (Acquisition  &  Management)  have  supported  the  use  of  parametric 
cost  estimating  techniques  (Spector,  1 995 : 1 ;  Druyun,  1 997 : 1 ).  Other  accomplishments 
include  the  preparation  and  distribution  of  the  Parametric  Cost  Estimating  Handbook,  the 
PCEI  newsletters,  recommendation  of  formal  training  requirements,  and  periodic 
workshops  (Hertling,  1998:16-19). 

The  reinvention  laboratories  were  sponsored  in  December  1995  by  the  Director  of 
DCAA  and  the  Commander  of  DCMC.  The  primary  objectives  of  the  labs  were  to 
expand,  test,  and  provide  feedback  and  recommendations  on  the  use  of  parametric 
estimating  (Hertling,  1998:20).  Thirteen  sites  were  chosen  to  form  integrated  product 
teams  (IPTs),  consisting  of  contractors,  major  buying  activities,  DCMC,  and  DCAA. 
These  sites  include  four  from  Boeing,  three  from  Lockheed  Martin,  two  from  Northrop, 
two  from  Raytheon,  GE  Aircraft  Engines,  and  Motorola  Space  &  Systems  (Hertling, 
1998:21).  The  1997  PCEI  Newsletter  advertises  successful  testing  of  the  use  of 
parametric  techniques  by  the  reinvention  labs.  “These  teams  are  demonstrating  that  the 
use  of  parametrics  results  in  improved  customer  satisfaction  through  better  estimates; 
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reduced  contract  award  cycle  time;  and  significant  reductions  in  proposal  preparation, 
evaluation,  and  negotiation  costs”  (Collins  and  Eck,  1997b:l). 

However,  as  of  May  1997,  there  had  been  no  procurements  at  the  Aeronautical 
Systems  Center,  Wright  Patterson  AFB  (WPAFB)  that  used  the  types  of  parametric 
techniques  promoted  by  the  PCEI  (Anderson,  1997:1).  If  parametric  estimating  is  more 
efficient  and  equally  accurate,  there  should  be  evidence  of  rapid  and  widespread  adoption 
of  its  use.  Yet  in  the  three  years  since  the  PCEI  was  formed,  there  have  been  no 
parametric  implementation  successes  at  a  major  contracting  center  of  the  Air  Force. 

* 

Summary 

Significant  effort  to  expand  the  use  of  parametric  estimating  into  DoD  contracting 
continues  because  the  current  processes  are  still  considered  inefficient.  It  is  also  believed 
that  parametric  estimating  is  a  better  way  of  preparing  an  estimate  combining  accuracy 
and  efficiency.  These  and  related  assertions  are  causing  the  expenditure  of  extensive 
resources  to  expand  the  role  of  parametric  estimating  in  the  contracting  arena.  If 
parametric  estimating  is  inappropriate  for  the  purposes  planned,  then  the  resources 
currently  being  devoted  to  implementing  it  are  being  wasted.  The  improper  use  of 
parametric  estimating  could  lead  to  inaccurate  estimates  causing  additional  wasted 
resources,  improperly  funded  programs,  and  associated  acquisition  perturbations. 

The  appropriate  cost  estimating  technique  according  to  the  situation  is  important 
because  the  best  methodology  should  lead  to  better  results  (i.e.  less  cost  estimation  error). 
Cost  estimates  are  one  of  the  keys  for  establishing  program  plans  of  execution.  While  it 
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is  difficult  to  place  an  exact  dollar  amount  on  the  value  of  a  good  estimate,  the 
importance  of  properly  performed  cost  estimates  at  all  stages  of  the  acquisition  cycle 
cannot  be  overstated.  The  purpose  of  this  study  is  to  evaluate  the  efficacy  of  parametric 
cost  estimating  by  comparing  its  perceived  utility  across  users  and  contractor  groups. 
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III.  Methodology 


Chapter  II  reviewed  the  literature  with  regard  to  how  cost  estimating  is  currently 
understood  and  accomplished.  Chapter  II  provided  generalizations  about  parametric 
estimating  and  the  selection  of  estimating  methods.  Chapter  II  also  introduced  the  PCEI 
and  presented  its  goals  and  objectives.  The  PCEI  advocates  a  position  that  parametrics 
can  not  only  shorten  cycle  time  but  also  provide  equivalent  as  accuracy  as  the  more 
conventional  bottoms-up  techniques.  The  literature  proposes  that  parametrics  is  primarily 
used  early  in  a  program  when  time  and  information  is  limited.  Whether  customer 
viewpoints  are  consistent  with  the  literature  or  the  PCEI  position  is  uncertain.  Research 
is  required  to  make  this  determination.  Chapter  III  contains  the  methodology  employed 
to  conduct  this  research. 

The  overall  objective  of  this  research  is  to  identify  differences  in  perspectives  about 
parametric  estimating  that  exist  between  important  users  of  parametric  techniques.  We 
need  to  know  if  PCEI  members  have  a  different  perspective  of  parametric  estimating  than 
buying  offices,  and  whether  industry  and  government  have  differing  views.  To 
accomplish  this,  a  qualitative  study  will  be  used. 

Qualitative  Research 

Research  Definition  and  Goals.  Qualitative  research  is  any  kind  of  research  that 
analyzes  and  produces  findings  about  data  that  cannot  be  expressed  in  quantitative  terms 
or  numbers  (Tesh,  1990:2-3).  Qualitative  research  is  mainly  concerned  with  the  nature  of 
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a  phenomena  putting  emphasis  on  processes  and  meanings  -  producing  more  detailed  data 
about  a  smaller  number  of  people  and  cases  (Labuschagne,  1996).  Qualitative  research 
seeks  to  describe  rather  than  quantify. 

Marshall  and  Rossman  establish  guidelines  for  qualitative  studies  (Marshall  and 
Rossman,  1991:Ch  3).  Included  is  research  that  delves  into  complexities  and  processes 
and  research  that  seeks  to  explore  where  and  why  policy  and  practice  do  not  work 
(Marshall  and  Rossman,  1991 :46).  Policy  makers  need  to  see  the  impact  of  their  policy. 
Qualitative  research  can  evaluate  the  problems,  identify  the  areas  that  can  be  influenced 
by  the  policy  maker,  and  show  the  consequences  of  policy  intervention  (Marshall  and 
Rossman,  1991:16). 

Research  can  be  classified  into  goals  of  exploration,  description,  prediction, 
explanation,  and  action  (Dane,  1990:5).  Exploratory  research  "involves  an  attempt  to 
determine  whether  or  not  a  phenomenon  exists"  (Dane,  1990:5).  Through  exploration, 
patterns  can  be  uncovered  as  well  as  concepts  more  clearly  developed.  One  purpose  is  to 
discover  important  questions,  processes,  and  relationships  (Marshall  and  Rossman, 

1991 :43).  Exploratory  research  can  also  identify  important  variables  for  future  research 
(Marshall  and  Rossman,  1991:15). 

Application  of  Research  to  Data  Gathering.  This  study  lends  itself  to  qualitative 
research.  This  research  is  an  exploratory  evaluation  of  the  perspectives  of  parametric 
estimating  within  the  defense  acquisition  community.  The  research  will  gather  data  to  try 
and  isolate  the  important  questions  about  parametric  usefulness.  Parametric  estimating 
has  been  established  as  a  very  broad  area  in  the  general  field  of  cost  estimating. 
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Parametric  estimating  involves  techniques  ranging  from  very  basic  to  very  complex. 
Exploratory  research  will  determine  whether  consensus  exists  between  differing  groups 
on  when  particular  applications  of  parametric  should  be  used  and  why.  For  example, 
there  is  not  a  current  source  of  information  that  stratifies  the  different  parametric 
techniques  according  to  different  phases  of  a  program  or  availability  of  information.  Data 
needs  to  be  gathered  on  the  subject  of  the  appropriateness  of  use  of  different  parametric 
applications. 

From  the  literature  review,  there  is  a  possible  difference  between  the  views 
advocated  by  the  PCEI  and  the  traditional  established  view  on  the  intended  purpose  for 
parametric  applications.  The  literature  suggests  using  parametrics  to  estimate  something 
which  has  never  been  built  before,  whereas,  the  PCEI  may  be  suggesting  that  parametrics 
could  be  a  better  way  to  estimate  across  the  entire  procurement  cycle.  Since  the  PCEI  is  a 
joint  government  and  industry  initiative,  participants  are  both  government  and  industry 
people.  Do  the  government  and  industry  PCEI  participants  actually  see  parametric 
estimating  in  a  similar  manner?  It  is  also  worthwhile  to  know  whether  non-PCEI 
government  and  industry  groups  see  parametric  estimating  similar  to  the  traditional  view 
or  more  consistent  with  the  PCEI  perspective.  Data  needs  to  be  gathered  on  the 
utilization  of  parametric  models  and  the  perceptions  of  the  usefulness  and  accuracy  of 
parametric  applications. 

In  general,  our  study  investigates  the  following  questions: 

1)  Has  the  use  of  parametric  estimating  increased  and  is  more  effort  needed  to 
increase  its  use? 
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2)  What  are  some  of  the  perceived  barriers  to  implementation? 

3)  Do  different  acquisition  groups  have  different  perceptions  on  the  accuracy  of 
parametrics? 

4)  How  useful  is  parametric  estimating  to  government  contracting  as  influenced  by 
phase  of  the  acquisition  cycle,  perceived  accuracy,  and  perceived  utility? 

5)  Is  there  a  difference  in  the  perceived  accuracy  and  usefulness  of  the  six  types  of 
parametric  applications? 

There  is  reason  to  believe  that  customers  may  have  a  different  view  of  using 
parametric  techniques  than  those  who  prepare  the  parametric  proposal  estimates.  The 
literature  suggests  that  there  is  possibly  a  relationship  between  the  customer's  degree  of 
acceptance  of  a  parametric  estimate  and  such  things  as:  the  customer's  perception  of  the 
estimate's  accuracy,  the  level  of  training  in  parametric  estimating  and  the  model  being 
used,  and  the  model's  credibility.  Data  needs  to  be  gathered  on  the  subject  of  customer 
perceptions  of  the  use  of  parametrics.  It  follows  that  a  comparison  of  PCEI  and 
customers  perceptions  would  also  be  very  worthwhile. 

Acquiring  an  understanding  of  what  is  really  going  on  in  something  as  complicated 
as  a  DOD  acquisition  streamlining  initiative  invariably  involves  more  than  just  reading  a 
few  documents.  A  boilerplate  label  like  "expand  the  use  of  parametric  estimating"  can  be 
deceiving  if  compared  to  actual  practice.  This  is  especially  true  if  one  were  tasked  to 
judge  the  goodness  or  benefit  that  a  streamlining  initiative  might  provide.  Information 
needs  to  be  gathered  on  the  subject  of  what  the  PCEI  is  really  doing,  not  at  the  summary 
level,  but  at  the  grassroots  level.  This  information  could  provide  a  scientifically  based 
determination  of  the  potential  benefits,  detriments,  and  impediments  of  this  change  in 
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how  we  price  DoD  acquisitions.  Questions  need  to  be  answered  like  what  basic 
differences  in  proposal  preparation  and  negotiation  processes  may  come  out  of  this 
initiative  and  what  gains  and/or  losses  might  those  changes  provide  to  the  acquisition 
process. 

Research  Design 

Review  of  Methodology.  The  research  strategy  to  gather  data  uses  a  mixture  of 
methodologies  characteristic  of  qualitative  research.  Cooper  and  Emory  list  several 
exploratory  techniques  which  can  be  relied  upon  to  gather  data.  These  include:  in-depth 
interviewing;  participant  observation;  films,  photographs,  and  videotape;  projective 
techniques  and  psychological  testing;  case  studies;  street  ethnography;  elite  interviewing; 
document  analysis;  and  proxemics  and  kinesics  (Cooper  and  Emory,  1995:1 18). 

Marshall  and  Rossman  also  include  questionnaires  and  surveys,  historical  analysis, 
content  analysis,  and  unobtrusive  measures  (Marshall  and  Rossman,  1991:83). 

This  study  will  combine  the  different  data  collection  techniques  involved  in 
historical  analysis,  questionnaires,  and  interviews.  Multiple  methods  will  be  used  to 
compensate  weaknesses  in  one  strategy  with  strengths  in  another.  While  not  a  perfect 
approach  given  the  exploratory  nature  of  this  study,  we  should  be  able  to  gain  valuable 
insight  into  the  underlying  value  of  parametric  techniques  as  they  apply  to  DoD 
contracting. 

Collection  of  Data.  Historical  analysis  is  "a  method  of  discovering,  from  records, 
what  happened  in  the  past"  and  is  particularly  useful  "for  establishing  a  baseline  or 


background"  (Marshall  and  Rossman,  1991 :95).  There  is  a  great  deal  of  information 
about  the  PCEI  which  can  provide  a  progression  of  the  PCEI  and  its  past  and  current 
accomplishments.  First  is  the  PCEI  newsletter  that  is  published  periodically  and  provides 
summary  level  information  on  accomplishments  at  specific  lab  sites.  This  information 
will  be  reviewed  and  analyzed. 

A  unique  opportunity  will  present  itself  in  early  March  1998.  The  PCEI,  in 
conjunction  with  the  International  Society  of  Parametric  Analyst,  will  hold  its  annual 
conference  in  Los  Angeles,  California.  Since  there  will  be  400  to  500  people  associated 
with  the  PCEI  at  the  conference,  this  will  be  an  excellent  opportunity  to  gather  an 
abundance  of  information  on  parametric  implementation  strategies  and  specific 
accomplishments.  We  will  use  this  conference  as  a  data  gathering  opportunity. 

A  compilation  of  PCEI  member  ideas,  attitudes,  and  perceptions  would  be 
invaluable.  The  best  way  to  accomplish  this  would  be  to  administer  a  questionnaire.  A 
questionnaire  is  administered  to  "learn  about  the  distribution  of  a  characteristic  or  set  of 
characteristics  or  a  set  of  attitudes  or  beliefs"  (Marshall  and  Rossman,  1991 :83).  The 
purpose  of  a  survey  is  to  "use  questionnaires  or  interviews  to  collect  data  from 
participants  in  a  sample  about  their  characteristics,  experiences,  and  opinions  in  order  to 
generalize  the  findings  to  a  population"  (Gall  and  others,  1996:289).  However,  the 
amount  of  time  available  until  the  conference  is  insufficient  to  properly  develop  a 
construct  valid  survey. 

Preparing  a  study  with  construct  validity  involves  accepted  ideas  such  as 
establishing  a  focus  group,  running  a  pilot  study,  and  validating  constructs  (Churchill, 
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1979:66).  In  lieu  of  developing  this  validated  survey  instrument,  the  researchers  prepared 
a  list  of  items  based  on  information  from  the  literature  review,  and  provided  the  items  to 
five  available  topic  experts  for  review.  Four  experts  were  at  Aeronautical  Systems  Center 
and  one  at  DCAA  headquarters.  The  topic  experts  were  requested  to  review  the  survey 
instrument  to  see  if:  the  questions  were  clearly  understood;  if  there  were  any  obvious 
errors;  and  if  there  was  anything  they  would  add  considering  the  topic  at  hand.  Errors 
were  corrected  and  items  considered  redundant  or  confusing  were  deleted  and/or 
corrected.  The  final  questionnaire  instrument  is  included  in  Appendix  A. 

The  questionnaire  will  be  taken  to  the  conference  and  announced  early  in  the  first 
day  of  the  two  day  conference.  Participation  will  be  strictly  voluntary.  Blank 
questionnaires  will  be  available  at  a  prime  location  near  the  entrance  to  the  conference 
ballroom.  Boxes  will  be  provided  for  completed  surveys  at  the  same  location. 

The  questionnaire  will  provide  a  needed  means  of  reaching  a  large  number  of  people 
on  some  of  the  obvious  questions.  The  conference  will  also  provide  the  opportunity  to 
conduct  interviews.  Because  of  the  compressed  and  intense  pace  of  the  conference,  it  is 
expected  that  the  number  of  interviews  will  be  relatively  small. 

The  questionnaire  will  attempt  to  gather  experience  data  on  five  basic  areas: 

1)  expanded  use  of  parametric  estimating,  2)  utility  (accuracy,  acquisition  phase,  contract 
type),  3)  parametric  applications,  4)  challenges  or  barriers,  and  5)  demographic 
information. 

Defining  the  Research  Concepts  to  be  Explored.  The  first  research  concept  to  be 
defined  is  that  of  expanded  use  of  parametric  estimating.  Nothing  is  clearer  about  the 
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stated  goal  of  the  PCEI  than  to  expand  the  use  of  parametric  estimating.  The  concept  of 
expanded  use  should  mean  that  in  contractor  proposals  for  use  in  establishing  firm 
contract  prices,  parametric  techniques  should  be  more  prevalent,  if  not  the  sole  means  of 
estimating  the  cost  or  price  of  the  proposal.  The  mention  of  firm  contract  proposals  is  in 
contrast  to  those  parametric  applications  that  are  used  to  derive  rough  order  of  magnitude 
(ROM)  or  budgetary  estimates  for  planning  purposes.  Firm  contract  proposals  does  not 
imply  firm  fixed  price  type  contracting  arrangements,  only  that  the  parametric  estimate 
was  derived  for  the  purpose  of  establishing  a  contract  price  for  some  provided  good  or 
service. 

Perhaps  the  most  difficult  concept  to  be  defined  in  this  research  exploration  is  that 
of  utility  of  parametric  estimates.  Utility  is  defined  as  fitness  for  some  purpose  or  worth 
to  some  end  (Webster,  1984:1300).  The  difficulty  in  applying  this  textbook  definition  is 
determining  the  fitness  or  worth  variables  and  their  relative  importance.  Utility  is 
therefore  a  perceived  value  in  something.  Not  only  will  there  be  different  perceptions  in 
overall  utility  of  something,  but  there  will  be  differences  in  perceptions  in  what  defines 
utility  in  relative  importance  of  the  attributes  being  considered.  For  example,  to  some  the 
utility  of  parametrics  may  be  based  on  how  accurately  it  derives  an  estimate,  to  others, 
how  easily  or  quickly  it  derives  an  estimate,  and  to  others,  how  versatile  parametric 
estimating  is  in  deriving  estimates  in  particular  situations.  In  Apgar’s  discussion  of 
advantages  and  disadvantages  of  conventional  and  parametric  estimating,  issues  such  as 
ease  of  use,  efficiency  of  use,  requirements  for  special  training,  amount  used,  degree 
understood  and  accepted,  and  amount  of  input  detail  required  are  proposed  (Parametric 
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Cost  Estimating  Course  Manual,  1987:126-129).  These  issues  help  to  identify  the 
variables  of  fitness  or  worth  in  Webster’s  definition  of  utility.  These  ideas  form  the  basis 
of  the  research  concept  of  utility  to  be  used  in  this  thesis.  In  this  exploratory  research,  the 
variables  of  utility  are  weighted  equally  in  developing  overall  utility  measures  since  there 
is  no  accepted  basis  to  provide  weightings.  This  research  will  investigate  the  concept  of 
utility  of  parametric  estimating  from  the  perspective  of  parametrics  as  an  aggregate  term 
and  from  the  perspective  of  specific  parametric  applications. 

This  research  has  stated  before  and  will  again,  parametric  estimating  is  an  all- 
encompassing  term  for  a  variety  of  techniques  that  develop  estimated  amounts  in  a 
manner  more  statistically  sound  than  just  experience  or  judgment  or  other  non- 
quantifiable  methods.  For  lack  of  any  better  structure,  this  research  has  already  classified 
parametric  estimates  according  to  six  types:  1)  CERs  that  are  cost-to-cost;  2)  CERs  that 
are  cost-to-non-cost;  3)  commercial  models  that  estimate  the  entire  cost  of  a  proposal;  4) 
commercial  models  that  estimate  only  significant  portions  of  a  proposal;  5)  in-house 
models  that  estimate  the  entire  cost  of  a  proposal;  and  6)  in-house  models  that  estimate 
only  significant  portions  of  a  proposal.  To  try  to  measure  the  utility  of  parametric 
estimating  as  a  general  practice  may  be  somewhat  in  error  because  the  utility  of  each  of 
the  applications  could  possibly  be  different.  This  research  will  explore  the  possibility  of 
this  by  seeking  responses  to  six  identical  questions  on  the  six  different  applications 
(Section  3  of  Appendix  A).  The  six  questions  deal  with  the  utility  aspects  of  efficiency  of 
use,  ease  of  use,  accuracy,  frequency  of  use,  and  versatility  of  use. 
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The  last  concept  to  be  explored  is  that  of  barriers  or  challenges  to  using  parametric 
estimating.  While  this  is  somewhat  redundant  to  the  concept  of  expanded  use,  it  differs 
because  it  offers  a  much  wider  variety  of  specific  possible  causes.  Many  of  the  potential 
causes  were  identified  by  the  PCEI  itself  and  some  are  developed  by  the  researchers 
based  on  experience  or  knowledge  gained  from  the  literature  review. 

Purposeful  Sampling.  While  the  questionnaire  instrument  does  not  meet 
requirements  to  establish  construct  validity  and  provide  data  from  which  quantitative 
conclusions  can  be  drawn,  it  does  provide  a  database  of  experience  data  that  includes 
demographic  differentiation.  Differing  cell  demographics  can  express  the  things  different 
groups  believe  to  be  important.  The  data  gathered  from  the  parametric  annual  conference 
will  reflect  the  perspective  of  the  PCEI  members,  because  the  attendees  have  some 
association  to  parametric  estimating. 

Gathering  similar  information  from  customer  oriented  acquisition  people  not 
associated  with  the  initiative  will  involve  providing  the  questionnaire  to  selected 
acquisition  personnel  who  have  familiarity  with  parametric  estimating.  Since  the  practice 
is  not  used  extensively  in  proposals  that  establish  firm  contract  prices,  the  population  of 
contracting  people  who  can  provide  knowledgeable  opinions  is  limited.  A  sample  will  be 
developed  based  on  the  researcher's  knowledge  of  contracting  officer  experiences. 
Additionally,  the  ASC  business  clearance  reviewers  are  knowledgeable  about  who  has 
been  involved  in  parametrically  based  proposal  efforts  and  their  input  will  be  sought  on 
appropriate  candidates  to  request  questionnaire  input.  The  researchers  consider  the  idea 
of  random  sampling  inappropriate  for  this  exploratory  research  since  the  objectives  are  to 
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derive  some  ideas  on  differences  in  perceptions  on  the  use  of  parametrics.  Someone  who 
does  not  know  what  parametric  estimating  is,  at  the  level  of  a  working  understanding, 
provides  no  contribution  to  the  accomplishment  of  this  objective.  Customer  inputs  will 
also  be  sought  from  ASC  Price  Analysts,  believed  to  be  as  knowledgeable  about  the 
subject  as  any  group  at  ASC.  It  is  also  important  to  gain  input  from  intermediate 
customers  like  Defense  Contract  Management  Command  (DCMC)  and  Defense  Contract 
Audit  Agency  (DCAA).  These  intermediate  customers  are  responsible  for  initial 
proposal  evaluation  and  approval  of  estimating  systems  that  produce  all  estimates. 

DCAA  and  DCMC  are  similar  to  ASC  in  that  the  actual  number  of  people  knowledgeable 
and  experienced  in  parametric  estimating  is  believed  to  be  fairly  small.  In  general,  the 
questionnaire  sample  size  of  non-PCEI  will  likely  not  be  very  large  since  the  researchers 
are  seeking  customers  with  a  working  understanding  of  parametric  estimating. 

Data  Analysis 

The  data  analyses  used  for  exploratory  research  tend  to  be  general.  Often  used  are 
qualitative  analyses  or  "nonnumerical  analyses  concerning  quality  rather  than  quantity" 
(Dane,  1990:235).  Qualitative  analysis  organizes  data  in  a  manner  to  discover  patterns, 
themes,  forms  and  qualities  found  in  the  raw  data  collected  (Labuschagne,  1996).  Gall, 
Borg  and  Gall  identify  several  techniques  known  collectively  as  "exploratory  data 
analysis,  which  is  a  method  for  discovering  unforeseen  or  unexpected  patterns  in  the  data 
and  consequently  for  gaining  new  insights  and  understanding  of  the  natural  phenomena" 


(Gall  and  others,  1996:197).  A  part  of  exploratory  data  analysis  is  the  use  of  descriptive 
statistics. 

Descriptive  statistics  are  "mathematical  techniques  for  organizing  and  summarizing 

a  set  of  numerical  data"  (Gall  and  others,  1996:175).  There  are  three  main  groups  of 

descriptive  statistics:  frequency  counts  and  frequency  distributions,  graphical 

representations  of  data,  and  summary  statistics  (Graziano  and  Raulin,  1989:87).  Graziano 

and  Raulin  summarize  the  three  groups  as  follows: 

Descriptive  statistics  help  us  to  summarize  and  describe  large 
quantities  of  scores  in  only  a  few  numbers.  They  are  a  vital  first 
step  in  interpreting  research  data.  Even  when  more  complicated 
research  designs  are  used,  describing  data  will  always  be  the  first 
step  in  any  data-analysis  procedures.  Frequency  distributions  and 
graphical  representations  of  data  are  often  helpful.  Summary 
statistics  simplify  data  further.  There  are  summary  statistics  to 
indicate  the  typical  score  (measures  of  central  tendency),  including 
the  mean,  the  median,  and  the  mode.  There  are  summary  statistics 
to  indicate  the  variability  of  the  scores,  including  the  range,  the 
variance,  and  the  standard  deviation.  Finally,  there  are  descriptive 
statistics  to  indicate  the  degree  of  relationship  between  two  or 
more  variables  (correlations).  (Graziano  and  Raulin,  1989:106) 

This  study  will  use  exploratory  data  analysis,  using  both  qualitative  analysis  and 

descriptive  statistics  techniques  to  evaluate  the  research  questions. 


Summary 

This  research  effort  is  designed  as  an  exploratory,  qualitative  study  concerning  the 
perspectives  of  parametric  estimating  within  the  defense  acquisition  community.  A 
mixed  methodology  of  historical  analysis,  questionnaires,  and  interviews  will  be  used  to 
determine  the  differences  in  perspectives  about  parametric  estimating  and  between  whom 
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the  differences  exist.  The  exploratory  data  analysis  techniques  used  will  assist  in  our 
investigation  of  the  research  questions. 
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IV.  Results  and  Analysis 


Introduction 

This  chapter  provides  the  results  of  the  data  collection  and  analysis  process  outlined 
in  Chapter  III.  An  overview  of  the  questionnaire  demographics  is  presented.  Results  are 
then  presented  and  discussed  in  detail. 


Overview  of  Questionnaire  Demographics 

The  questionnaire  results  were  categorized  into  four  acquisition  groups:  PCEI  Industry, 
PCEI  Government,  Non-PCEI  Industry,  and  Non-PCEI  Government.  Figure  1  provides 
the  total  returns  broken  out  by  acquisition  groups. 


The  results  from  the  PCEI  industry  and  government  acquisition  groups  were 
received  in  March  1998  during  the  PCEI  Conference.  Fifty-seven  questionnaires  were 
returned  during  the  conference.  One  questionnaire  was  discarded  because  no 
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demographic  information  was  provided.  Ten  other  questionnaires  contained  some 
missing  data  but  were  used  when  possible  to  answer  the  investigative  questions.  Thirty- 
four  questionnaires  were  analyzed  from  the  PCEI  industry  acquisition  group  and  22  from 
the  PCEI  government  acquisition  group. 

A  total  of  3 1  questionnaires  were  received  from  the  Non-PCEI  acquisition  group 
during  June  1998.  Two  were  discarded  when  no  demographic  information  was  provided. 
Two  other  questionnaires  contained  missing  information,  but  like  above,  were  used  when 
possible  to  answer  the  investigative  questions.  For  the  Non-PCEI  acquisition  group,  10 
questionnaires  were  analyzed  from  industry  and  19  questionnaires  from  government. 

While  other  demographic  information  was  requested,  the  only  information  used  was 
to  distinguish  between  the  four  parametric  acquisition  groups.  Since  this  is  a  qualitative 
study,  no  attempt  was  made  to  correlate  any  of  the  demographic  information  with  the 
item  responses. 

Questionnaire  Overview  and  Results 

The  questionnaire  included  five  sections:  1)  Expanded  use  of  Parametric  Estimating, 
2)  Utility  -  Accuracy,  Acquisition  Phase,  Contract  Type,  3)  Parametric  Applications,  4) 
Main  Challenges  or  Barriers,  and  5)  Demographic  Information.  The  questionnaire  is 
included  in  Appendix  A.  Sections  1-3  were  designed  using  a  five  point  likert  scale  from 
“Strongly  Disagree”  to  “Strongly  Agree.”  Items  from  these  sections  were  used  to  answer 
investigative  questions  one,  and  three  through  five.  Several  items  were  designed  to  be 
answered  in  the  opposite  direction  and  those  will  be  identified  with  the  applicable 
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investigative  question.  The  responses  concerning  investigative  question  two  conies  from 
Section  4  of  the  questionnaire  and  does  not  use  the  likert  scale.  Under  each  investigative 
question,  “S”  refers  to  the  section  of  the  questionnaire  and  “Q”  refers  to  the  question 
number. 

Investigative  Question  One:  Has  the  use  of  parametric  estimating  increased  and  is 
more  effort  needed  to  increase  its  use?  Investigative  question  one  looks  at  both  the  actual 
increase  in  use  of  parametric  techniques  and  whether  more  effort  is  still  needed  to 
increase  the  use  of  parametric  estimating.  The  intent  of  the  first  part  of  investigative 
question  one  is  to  evaluate  if  the  four  acquisition  groups  perceive  an  increase  in  the  use  of 
parametric  techniques.  The  second  part  addresses  whether  the  acquisition  groups  feel 
more  effort  is  needed  to  increase  the  use  of  parametric  techniques.  The  following  items 
from  the  questionnaire  were  used  to  evaluate  whether  there  has  been  an  increase  in  use  of 
parametric  techniques. 

S1Q1 :  The  use  of  parametric  techniques  has  increased  in  the  past  3  years. 

S 1 Q2:  The  government  and  industry  have  been  successful  in  expanding  the  use  of 
parametric  estimation. 

Table  1  provides  the  results  for  these  questions. 

For  S1Q1,  the  data  indicates  a  large  majority  of  the  respondents  either  agree  or 
strongly  agree  that  there  has  been  increased  use  of  parametric  techniques  within  the  past 
three  years,  ranging  from  74  percent  for  Non-PCEI  Government  to  90  percent  for  Non- 
PCEI  Industry.  The  PCEI  groups  responded  with  85  percent  and  77  percent  agreement  for 
industry  and  government,  respectively.  For  S1Q2,  only  Non-PCEI  Government  had  a 
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lower  than  60  percent  agreement  that  government  and  industry  have  been  successful  in 
expanding  the  use  of  parametrics.  Only  32  percent  of  Non-PCEI  Government  responded 
positively  to  this  question. 


Table  1.  Responses  to  Questionnaire  Items  for  Investigative  Question  One 


(Has  use  increased?). 


Item 

n 

Strongly 

Disagree 

Disagree 

Neither 

Agree 

Strongly 

Agree 

Mean 

PCEI  -Industry 

4.00 

SI  Q1 

34 

0.000 

0.000 

0.147 

0.441 

0.412 

4.26 

SI  Q2 

34 

0.000 

0.059 

0.265 

0.559 

0.118 

3.74 

PCEI -Govt 

3.73 

SI  Q1 

22 

0.045 

0.045 

0.136 

0.545 

0.228 

3.82 

SI  Q2 

22 

0.045 

0.091 

0.182 

0.545 

0.136 

3.64 

Non  PCEI  -  Industry 

3.65 

SI  Q1 

10 

0.000 

0.100 

0.000 

0.700 

0.200 

4.00 

SI  Q2 

10 

0.000 

0.300 

0.100 

0.600 

0.000 

3.30 

Non  PCEI -Govt 

■ 

SI  Q1 

19 

0.000 

0.053 

0.211 

0.579 

0.158 

1 

SI  Q2 

19 

0.000 

0.211 

0.474 

0.316 

0.000 

1 

The  following  items  from  the  questionnaire  were  used  to  evaluate  if  more  effort  is 
needed  to  increase  the  use  of  parametric  techniques. 

S 1 Q3 :  More  effort  is  needed  by  industry  to  expand  the  use  of  parametric  estimating. 

S1Q4:  More  effort  is  needed  by  the  Government  to  expand  the  use  of  parametric 

estimating. 

S 1 Q5 :  The  use  of  parametric  estimating  techniques  should  be  expanded  in  DoD 

contracts. 

Table  2  provides  the  results  for  this  question. 

The  data  shows  that,  except  for  Non-PCEI  Government,  there  is  very  strong 
agreement  that  more  effort  is  needed  to  expand  the  use  of  parametric  estimating  and  the 
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use  should  be  expanded  in  DoD  contracts.  Except  for  Non-PCEI  Government,  the  level 


of  agreement  is  nearly  or  greater  than  90%.  The  Non-PCEI  Government  responses 
showed  63%  agreement  that  more  effort  is  needed,  but  only  37%  agreed  that  the  use  of 
parametric  estimating  should  be  expanded  in  DoD  contracts. 


Table  2.  Responses  to  Questionnaire  Items  for  Investigative  Question  One 


(Is  more  effort  needed?) 


Item 

n 

Strongly 

Disagree 

Disagree 

Neither 

Agree 

Strongly 

Agree 

Mean 

PCEI  -Industry 

■nn 

SI  Q3 

34 

0.000 

0.000 

0.029 

0.412 

0.559 

SI  Q4 

34 

0.000 

0.000 

0.059 

0.353 

0.588 

H9 

SI  Q5 

34 

0.000 

0.000 

0.029 

0.471 

0.500 

4.47 

PCEI -Govt 

4.44 

SI  Q3 

22 

0.000 

0.045 

0.091 

0.364 

0.500 

4.45 

SI  Q4 

22 

0.000 

0.000 

0.000 

0.455 

0.545 

4.55 

SI  Q5 

22 

0.000 

0.000 

0.091 

0.500 

0.409 

4.32 

Non  PCEI  -  Industry 

4.31 

SI  Q3 

10 

0.000 

0.000 

0.100 

0.700 

0.200 

4.10 

SI  Q4 

10 

0.000 

0.000 

0.000 

0.600 

0.400 

4.40 

SI  Q5 

10 

0.000 

0.000 

0.000 

0.556 

0.444 

4.44 

Non  PCEI  -  Govt 

3.54 

SI  Q3 

19 

0.000 

0.105 

0.263 

0.526 

0.105 

3.63 

SI  Q4 

19 

0.000 

0.105 

0.263 

0.579 

0.053 

3.58 

SI  Q5 

19 

0.000 

0.053 

0.579 

0.263 

0.105 

3.42 

A  comparison  of  the  mean  scores  for  has  use  increased  (S1Q1  and  S1Q2  combined) 
and  is  more  effort  needed  (S1Q3,  S1Q4,  and  S1Q5  combined)  is  shown  in  Figure  2.  A 
mean  score  of  3.0  means  neither  agree  nor  disagree,  4.0  means  agree,  and  5.0  means 
strongly  agree.  All  four  groups  had  a  mean  score  above  the  neutral  position,  reflecting 
some  level  of  agreement  that  the  use  of  parametrics  has  increased  and  that  there  is  a  need 
for  more  effort  to  expand  the  use  of  parametrics.  The  responses  were  not  strongly  in 
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favor  of  that  suggestion,  just  above  the  neutral  response. 


Has  Use  Increased 
Is  More  Effort  Needed 


Figure  2.  Mean  Scores  -  Has  Use  Increased  and  Is  More  Effort  Needed? 

Both  PCEI  groups  and  the  Non-PCEI  industry  group  have  similar  responses 
concerning  the  increased  use  of  parametrics  and  whether  more  effort  is  needed  to  expand 
the  use  of  parametrics  in  DoD  contracts.  The  Non-PCEI  government  group  had  the 
lowest  response  score  in  both  categories  but  was  only  noticeably  lower  about  whether 
more  effort  is  needed. 

Investigative  Question  Two:  What  are  some  of  the  perceived  barriers  to 
implementation?  Many  potential  barriers  have  been  discussed  by  the  PCEI.  The  intent  of 
investigative  question  two  is  to  evaluate  how  the  different  acquisition  groups  perceive  the 
barriers  to  expand  the  use  of  parametrics. 

Section  4  of  the  questionnaire  was  used  to  identify  the  main  challenges  or  barriers  to 
parametric  estimating.  Respondents  were  requested  to  identify  the  top  5  challenges  or 
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barriers.  Since  not  all  respondents  were  consistent  in  ranking  the  challenges  1  through  5, 
the  challenges  or  barriers  identified  were  merely  counted  the  number  of  times  they  were 
identified  as  barriers.  Table  3  provides  the  total  count  that  each  parametric  barrier  was 
listed  in  the  top  5. 


Table  3.  Responses  to  Questionnaire  Items  for  Investigative  Question  Two 


Number  of  Times  Noted  as  a  Parametric  Implementation  Barrier 

(Combined  TINA  &  Acquisition  Rules  and  Utility  &  Difficulty  of  Use) 

PCEI 

PCEI 

Non-PCEI 

Non-PCEI 

Industry 

Government 

Industry 

Government 

Data 

29 

16 

5 

11 

Training 

26 

20 

5 

10 

Accuracy 

7 

8 

5 

14 

Risk  Assessment 

8 

5 

4 

6 

TINA/Acquisition  Rules 

15 

7 

14 

9 

Current  Bus  Environment 

11 

7 

2 

5 

Other 

5 

4 

1 

0 

Lack  of  Govt  Support 

23 

7 

9 

8 

Lack  of  Industry  Support 

21 

7 

1 

7 

Utility/Usefulness  Difficulty 

6 

11 

2 

18 

Model  Development 

23 

12 

4 

14 

Total  Responses 

174 

104 

52 

102 

The  raw  data  of  Table  3  is  revised  in  Table  4  to  provide  a  ranking  system  of  the 
barriers  to  expanded  implementation  of  parametric  estimating.  Three  categories,  risk 
assessment,  current  business  environment,  and  other,  were  deleted  because  they  were  not 
rated  highly  by  any  of  the  four  acquisition  groups.  The  Truth  In  Negotiation  Act  (TINA) 
and  acquisition  rules  were  considered  similar  enough  to  be  combined  into  one  category  as 
were  utility  and  difficulty  of  use.  The  deletion  and  combination  of  items  allows  a  better 
visualization  of  those  potential  barrier  causes  that  can  be  considered  important  to  the 


46 


different  groups.  The  Mean  Ranking  is  the  average  rank  of  all  four  acquisition  groups 
and  Rank  is  the  ranking  of  the  Mean  Ranking.  The  results  are  shown  in  Table  4. 


Table  4.  Ranking  Responses  to  Questionnaire  Items  for  Investigative  Question  Two 


Ranking  Responses  of  Parametric  Implementation  Barriers 

PCEI 

Industry 

PCEI 

Govt 

Non-PCEI 

Industry 

Non-PCEI 

Govt 

Mean 

Ranking 

Rank 

Data 

1 

2 

3 

4 

2.5 

1 

Training 

2 

1 

3 

5 

2.8 

2 

Accuracy 

9 

5 

3 

2 

4.8 

5 

TINA/Acquisition  Rules 

6 

6 

1 

6 

4.8 

5 

Lack  of  Govt  Support 

3 

6 

2 

7 

4.5 

4 

Lack  of  Industry  Support 

5 

6 

10 

8 

7.3 

8 

Utility/Usefulness  Difficulty 

10 

4 

8 

1 

5.8 

7 

Model  Development 

3 

3 

6 

2 

3.5 

3 

The  data  on  barriers  can  be  interpreted  based  on  both  the  average  of  all  the  group 
responses  and  on  the  differences  between  acquisition  group  responses.  The  data  provides 
findings  that  are  both  intuitive  and  revealing. 

All  four  acquisition  groups  identified  data  and  training  concerns  as  two  of  their  top 
five  challenges  or  barriers  to  implementing  parametric  estimating.  The  PCEI  forecasted 
the  importance  of  training  to  the  parametric  initiative,  and  training  is  one  of  the  top 
ranked  barriers  from  every  group  except  Non-PCEI  Government.  A  possible  explanation 
for  the  low  ranking  of  training  by  Non-PCEI  Government  may  be  the  belief  that  the 
government  is  nearly  adequately  trained  to  deal  with  parametric  estimates,  or  that  training 
is  available  and  will  be  obtained  when  needed  based  on  the  demands  of  the  proposal 
situation.  The  high  ranking  of  data  and  model  development  is  something  of  a  revelation. 
Apparently,  the  difficulty  of  obtaining  and  organizing  the  correct  data  and  developing 
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parametric  models  with  that  data  is  a  tougher  job  than  originally  envisioned  or  currently 
acknowledged  because  it  was  not  one  of  the  original  barriers  repeatedly  discussed  by  the 
PCEI  in  the  newsletter. 

There  are  other  noteworthy  observations,  most  dealing  with  one  or  two  group's 
difference  in  perceptions  from  other  groups.  The  most  glaring  example  is  the  Non-PCEI 
government  and  industry  group  perceptions  of  accuracy  compared  with  PCEI  government 
and  industry  groups.  The  PCEI  confidence  in  the  accuracy  of  parametric  techniques  does 
not  seem  to  be  shared  by  Non-PCEI  groups.  This  will  be  discussed  more  in  the  sections 
of  this  thesis  pertaining  to  investigative  questions  three  and  five. 

Another  example  of  group  differences  is  the  Non-PCEI  Government  high  ranking  for 
utility/usefulness/difficulty  of  use.  The  utility  construct  was  defined  in  very  broad  terms 
in  Chapter  III  and  dealt  with  issues  like  acquisition  phase,  contract  type,  and  ease  of  use. 
A  discussion  of  the  Non-PCEI's  perceptions  of  utility  are  addressed  in  the  sections 
pertaining  to  investigative  questions  four  and  five. 

The  last  finding  on  barriers  concerns  the  Non-PCEI  Industry's  high  ranking  of 
TINA/FAR  rules  and  lack  of  government  support.  Section  4  of  the  survey  provided  the 
opportunity  for  respondents  to  write  in  barriers  to  expanded  parametric  use.  A  popular 
response  was  the  FAR  15  requirement  for  subcontractor  cost  and  pricing  data.  Non-PCEI 
Industry  may  perceive  the  government  has  an  aversion  to  using  parametric  techniques  to 
estimate  the  cost  for  subsystems  that  are  purchased  items.  Government  staff  personnel 
who  provide  approval  authority  to  negotiate  and  finalize  procurement  actions  may  still 
feel  bound  by  what  they  believe  to  be  hard  requirements  for  subcontract  pricing  data. 
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This  may  be  the  cause  of  the  high  ranking  for  lack  of  government  support  by  Non-PCEI 
Industry  people,  although  other  reasons  may  exist. 

Investigative  Question  Three:  Do  different  acquisition  groups  have  different 
perceptions  on  the  accuracy  of  parametrics?  The  intent  of  investigative  question  three  is 
to  determine  if  parametric  estimating  techniques  are  considered  as  accurate  as  other  cost 
estimating  techniques.  The  following  survey  items  were  used  to  determine  the  parametric 
users’  view  of  accuracy: 

S 1 Q9 :  The  need  for  accuracy  limits  expansion  of  parametric  techniques. 

S2Q1 0:  Parametric  estimates  are  as  accurate  as  traditional  cost  build  up  approaches  on 

concept/budgetary  estimates. 

S2Q1 1 :  Parametric  estimates  are  as  accurate  as  traditional  cost  build  up  approaches  on 
mature  program  estimates. 

S2Q12:  Parametric  estimates  are  as  accurate  as  traditional  cost  build  up  approaches  in 

forecasting  production  costs. 

S3Q5 :  (six  separate  questions)  CERs  that  are  Cost-to-Cost  provide  benefits  in 

accuracy;  CERs  that  are  Cost-to-Non-Cost  provide  benefits  in  accuracy; 
Commercial  Models  that  estimate  the  entire  cost  of  a  proposal  provide 
benefits  to  accuracy;  Commercial  Models  that  estimate  only  significant 
portions  of  a  proposal  provide  benefits  to  accuracy;  In-house  Models  that 
estimate  only  significant  portions  of  a  proposal  provide  benefits  to  accuracy; 
In-house  Models  that  estimate  the  entire  cost  of  a  proposal  provide  benefits  to 
accuracy. 

Table  5  provides  the  results  for  these  questions. 

Item  S1Q9  was  inadvertently  designed  to  be  answered  in  the  opposite  direction, 
meaning,  a  higher  response  approaching  five  reflects  negatively  on  parametric  estimating. 
The  overall  mean  score  for  each  of  the  four  acquisition  groups  reflects  a  recoding  of  the 
data  for  item  S1Q9. 
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Table  5.  Responses  to  Questionnaire  Items  for  Investigative  Question  Three 


Strongly 

Strongly 

Item 

n 

Disagree 

Disagree 

Neither 

Agree 

Agree 

Mean 

PCEI  -Industry 

3.89 

SI  Q9 

34 

0.353 

0.382 

0.088 

0.118 

0.059 

2.15 

S2Q10 

34 

0.000 

0.059 

0.059 

0.471 

0.412 

4.24 

S2Q11 

34 

0.000 

0.029 

0.118 

0.529 

0.324 

4.15 

S2Q12 

34 

0.000 

0.029 

0.118 

0.529 

0.324 

4.15 

S3  Q5  (CER  Cost  to  Cost) 

31 

0.000 

0.129 

0.194 

0.516 

0.161 

3.71 

S3  Q5  (CER  Cost  to  NonCost) 

32 

0.000 

0.094 

0.250 

0.469 

0.188 

3.75 

S3  Q5  (Commercial -entire  proposal) 

33 

0.030 

0.030 

0.333 

0.455 

0.152 

3.67 

S3  Q5  (Commercial -portion) 

30 

0.000 

0.033 

0.233 

0.533 

0.200 

3.90 

S3  Q5  (In-house-portion) 

33 

0.000 

0.030 

0.333 

0.485 

0.152 

3.76 

S3  Q5  (In-house-entire  proposal) 

32 

0.000 

0.031 

0.375 

0.406 

0.188 

3.75 

PCEI  -  Govt 

3.45 

SI  Q9 

22 

0.318 

0.318 

0.136 

0.091 

0.136 

2.55 

S2Q10 

22 

0.000 

0.045 

0.409 

0.455 

0.091 

3.59 

S2Q11 

22 

0.000 

0.045 

0.364 

0.500 

0.091 

3.64 

S2Q12 

22 

0.000 

0.091 

0.318 

0.455 

0.136 

3.64 

S3  Q5  (CER  Cost  to  Cost) 

18 

0.000 

0.056 

0.389 

0.444 

0.111 

3.71 

S3  Q5  (CER  Cost  to  NonCost) 

18 

0.056 

0.056 

0.444 

0.389 

0.056 

3.41 

S3  Q5  (Commercial-entire  proposal) 

17 

0.000 

0.176 

0.647 

0.118 

0.059 

3.13 

S3  Q5  (Commercial-portion) 

17 

0.000 

0.118 

0.647 

0.176 

0.059 

3.25 

S3  Q5  (In-house-portion) 

16 

0.000 

0.125 

0.563 

0.250 

0.063 

3.33 

S3  Q5  (In-house-entire  proposal) 

16 

0.000 

0.063 

0.688 

0.188 

0.063 

3.33 

Non  PCEI  -  Industry 

3.37 

SI  Q9 

10 

0.200 

0.200 

0.300 

0.300 

0.000 

2.70 

S2Q10 

10 

0.000 

0.200 

0.100 

0.700 

0.000 

3.50 

S2Q11 

10 

0.000 

0.200 

0.300 

0.400 

0.100 

3.40 

S2Q12 

10 

0.000 

0.100 

0.200 

0.600 

0.100 

3.70 

S3  Q5  (CER  Cost  to  Cost) 

8 

0.000 

0.125 

0.125 

0.500 

0.250 

3.88 

S3  Q5  (CER  Cost  to  NonCost) 

8 

0.000 

0.500 

0.000 

0.375 

0.125 

3.13 

S3  Q5  (Commercial-entire  proposal) 

9 

0.000 

0.111 

0.333 

0.556 

0.000 

3.44 

S3  Q5  (Commercial-portion) 

8 

0.000 

0.125 

0.625 

0.250 

0.000 

3.13 

S3  Q5  (In-house-portion) 

9 

0.000 

0.111 

0.556 

0.333 

0.000 

3.22 

S3  Q5  (In-house-entire  proposal) 

8 

0.000 

0.375 

0.250 

0.375 

0.000 

3.00 

Non  PCEI  -  Govt 

2.94 

SI  Q9 

34 

0.000 

0.211 

0.421 

0.211 

0.158 

3.32 

S2Q10 

34 

0.000 

0.105 

0.368 

0.526 

0.000 

3.42 

S2Q11 

34 

0.053 

0.263 

0.211 

0.474 

0.000 

3.11 

S2Q12 

34 

0.000 

0.211 

0.526 

0.263 

0.000 

3.05 

S3  Q5  (CER  Cost  to  Cost) 

31 

0.000 

0.211 

0.368 

0.316 

0.105 

3.32 

S3  Q5  (CER  Cost  to  NonCost) 

32 

0.000 

0.158 

0.526 

0.263 

0.053 

3.21 

S3  Q5  (Commercial-entire  proposal) 

33 

0.158 

0.158 

0.579 

0.105 

0.000 

2.63 

S3  Q5  (Commercial-portion) 

30 

0.053 

0.368 

0.474 

0.105 

0.000 

2.63 

S3  Q5  (In-house-portion) 

33 

0.053 

0.263 

0.579 

0.105 

0.000 

2.74 

S3  Q5  (In-house-entire  proposal) 

32 

0.053 

0.368 

0.474 

0.105 

0.000 

2.63 
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Using  S1Q9  alone,  the  data  indicates  the  four  acquisition  groups  hold  differing 
views  about  the  accuracy  of  parametric  estimating  techniques.  Seventy-four  percent  of 
the  PCEI  Industry  respondents  felt  the  need  for  accuracy  did  not  limit  the  expansion  of 
parametric  techniques.  In  contrast,  only  21  percent  of  Non-PCEI  Government,  40  percent 
of  Non-PCEI  Industry,  and  64  percent  of  PCEI  Government  shared  this  view. 

S2Q10, 11,  and  12  address  accuracy  as  it  relates  to  the  various  stages  of  the 
acquisition  cycle.  The  results  provide  a  similar  picture  to  S1Q9.  Of  PCEI  Industry,  85  to 
89  respondents  felt  parametrics  were  as  accurate  as  traditional  techniques  during  all 
acquisition  phases.  In  contrast  only  26  to  53  percent  of  Non-PCEI  Government,  50  to  70 
percent  of  Non-PCEI  Industry,  and  55  to  59  percent  of  PCEI  Government  shared  this 
view. 

The  PCEI  Government  group  did  not  view  parametric  estimating  accuracy  as 
favorably  as  the  PCEI  Industry  group.  Sixty-four  percent  felt  the  need  for  accuracy  did 
not  limit  the  expansion  of  parametric  techniques  while  23  percent  did  see  it  as  limiting. 
While  between  55  to  60  percent  of  PCEI  Government  felt  parametric  techniques  were  as 
accurate  as  traditional  cost  build  up  approaches,  the  percentage  agreement  dropped  when 
looking  at  the  benefits  of  accuracy  from  the  six  different  types  of  parametrics.  The 
specific  differences  in  the  six  types  will  be  discussed  in  response  to  investigative  question 
five. 

The  Non-PCEI  Industry  held  similar  views  to  the  PCEI  Government.  Although  40 
percent  of  this  group  felt  the  need  for  accuracy  did  not  limit  the  expansion  of  parametric 
estimating  techniques,  30  percent  did  feel  the  need  for  accuracy  was  limiting  the 


51 


expansion.  Between  50  to  70  percent  of  Non-PCEI  Industry  felt  parametric  techniques 
were  as  accurate  as  traditional  cost  build  up  approaches.  Once  again,  the  percentage 
agreement  dropped  when  looking  at  the  benefits  of  accuracy  from  the  six  different  types 
of  parametric  estimates. 

The  Non-PCEI  Government  held  the  least  favorable  views  concerning  the  accuracy 
of  parametric  estimates.  Only  21  percent  of  Non-PCEI  Government  respondents 
indicated  the  need  for  accuracy  did  not  limit  the  expansion  of  parametric  estimating 
techniques,  while  37  percent  indicated  the  need  for  accuracy  did  limit  the  expansion. 
Forty-two  percent  were  neutral  on  this  item.  Of  Non-PCEI  Government  respondents, 
only  26  to  53  percent  felt  parametric  techniques  were  as  accurate  as  traditional  cost  build 
up  approaches  with  the  higher  percentage  agreement  during  the  concept/development 
phase  of  a  program.  Like  the  PCEI  Government  and  Non-PCEI  Industry  groups,  the 
percentage  agreement  dropped  when  looking  at  the  benefits  of  accuracy  from  the  six 
different  types  of  parametric  estimates.  However,  the  agreement  percentages  were  much 
lower  than  the  other  groups. 

A  comparison  of  the  mean  scores  for  the  perceptions  of  accuracy  for  the  acquisition 
groups  is  shown  in  Figure  3.  The  results  show  that  PCEI  Industry  has  the  highest  belief 
in  the  accuracy  of  parametrics,  Non-PCEI  Government  the  lowest,  and  the  other  two 
groups  nearly  the  same  halfway  between.  Some  possible  reasons  for  the  highest  and 
lowest  perceptions  of  accuracy  include  knowledge,  or  lack  of  knowledge,  of  the  effort 
devoted  to  developing  the  parametric  technique  and  therefore  its  statistical  predictive 
accuracy.  Perhaps  the  low  government  perception  is  due  to  a  lack  of  knowledge  on 
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ACCURACY  PERCEPTIONS 


PCEt  Industry  PCEI  Government  Non-PCEI  Industry  Non-PCEI  Government 


Figure  3.  Perceptions  of  Accuracy 

statistical  predictive  techniques  in  general.  The  difference  in  accuracy  perceptions  is 
significant  but  somewhat  incomplete  since  it  addresses  parametrics  in  general  without 
regard  to  type  of  parametric  application  under  consideration.  Accuracy  with  regard  to 
type  of  parametric  application  is  addressed  in  the  discussion  of  investigative  question 
five. 

What  may  be  more  important  than  the  perceptions  of  accuracy  is  the  impact  these 
perceptions  have  on  customer  acceptance.  The  two  are  likely  closely  related. 

Investigative  Question  Four:  How  useful  is  parametric  estimating  to  government 
contracting  as  influenced  bv  phase  of  the  acquisition  cycle,  perceived  accuracy,  and 
information  availability?  The  intent  of  investigative  question  four  is  to  evaluate  if 
parametric  techniques  are  considered  useful.  Usefulness  is  defined  in  terms  of  flexibility 
during  certain  phases  of  the  acquisition  cycle  and  contract  types,  and  whether  cost 
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information  availability  is  a  determinant.  The  following  survey  items  were  used  to 

determine  the  usefulness  of  parametric  estimating  techniques: 

S 1 Q5 :  The  use  of  parametric  estimating  techniques  should  be  expanded  in  DoD 

contracting. 

S2Q1 :  Parametric  estimating  should  only  be  used  on  cost  type  contracts. 

S2Q3:  Parametric  estimating  is  more  appropriately  used  during  the  concept/ 

development  phase  of  a  program. 

S2Q6:  If  sufficient  cost  information  is  available,  parametrics  should  not  be  used. 

S2Q7:  Contract  type  is  a  consideration  for  when  parametrics  should  be  used. 

S2Q8:  Parametrics  should  only  be  used  to  prepare  initial  budgetary  or  rough 

estimates. 

S2Q13:  Parametric  estimating  is  inappropriate  for  use  on  follow-on  production. 

Table  6  provides  the  results  for  these  questions. 

All  items  were  designed  to  be  answered  in  the  opposite  direction  except  S1Q5.  The 
overall  mean  scores  for  each  of  the  four  acquisition  groups  reflect  a  recoding  of  the  data 
for  the  opposite  direction  items. 

For  S1Q5,  over  90  percent  of  both  PCEI  acquisition  groups  and  Non-PCEI  Industry 
responded  that  the  use  of  parametric  techniques  should  be  expanded,  while  Non  PCEI- 
Govemment  indicated  only  37  percent  agreement  with  this  item.  S2Q1  and  S2Q7  deal 
with  considerations  concerning  the  type  of  contract.  For  S2Q1,  no  group  strongly 
supported  agreement  with  limiting  the  use  of  parametric  estimating  to  cost  type  contracts. 
This  may  be  somewhat  surprising  until  you  consider  CER’s  are  a  type  of  parametric 
estimate.  For  S2Q7,  the  highest  support  for  contract  type  being  a  consideration  for 
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Table  6.  Responses  to  Questionnaire  Items  for  Investigative  Question  Four 


Strongly  Strongly 


Item 

n 

Disagree 

Disagree 

Neither 

Agree 

Agree 

Mean 

PCEI  -Industry 

4.01 

S1Q5 

34 

0.000 

0.000 

0.029 

0.471 

0.500 

4.47 

S2Q1 

34 

0.441 

0.412 

0.088 

0.059 

0.000 

1.76 

S2  Q3 

34 

0.147 

0.294 

0.353 

0.147 

0.059 

2.68 

S2  Q6 

34 

0.324 

0.471 

0.176 

0.029 

0.000 

1.91 

S2  Q7 

34 

0.118 

0.529 

0.294 

0.059 

0.000 

2.29 

S2  Q8 

34 

0.441 

0.471 

0.029 

0.029 

0.029 

1.74 

S2Q13 

34 

0.353 

0.500 

0.000 

0.059 

0.088 

2.03 

PCEI  -  Govt 

SI  Q5 

22 

0.000 

0.000 

0.091 

0.500 

0.409 

4.32 

S2Q1 

22 

0.545 

0.364 

0.091 

0.000 

0.000 

S2  Q3 

22 

.0136 

0.591 

0.136 

0.091 

0.045 

2.41 

S2  Q6 

22 

0.136 

0.591 

0.227 

0.045 

0.000 

2.18 

S2  Q7 

22 

0.227 

0.455 

0.318 

0.000 

0.000 

2.09 

S2  Q8 

22 

0.409 

0.455 

0.136 

0.000 

0.000 

1.77 

S2Q13 

22 

0.500 

0.364 

0.091 

0.045 

0.000 

1.82 

Non  PCEI  -  Industry 

3.68 

SI  Q5 

10 

0.000 

0.000 

0.000 

0.556 

0.444 

4.44 

S2Q1 

10 

0.300 

0.200 

0.100 

0.400 

0.000 

2.60 

S2  Q3 

10 

0.000 

0.400 

0.300 

0.300 

0.000 

2.90 

S2  Q6 

10 

0.300 

0.200 

0.300 

0.200 

0.000 

2.40 

S2  Q7 

10 

0.200 

0.000 

0.600 

0.100 

0.100 

2.90 

S2  Q8 

10 

0.300 

0.500 

0.200 

0.000 

0.000 

1.90 

S2Q13 

10 

0.300 

0.400 

0.300 

0.000 

0.000 

2.00 

Non  PCEI  -  Govt 

3.18 

SI  Q5 

19 

0.000 

0.053 

0.579 

0.263 

0.105 

3.42 

S2Q1 

19 

0.053 

0.632 

0.263 

0.053 

0.000 

2.32 

S2  Q3 

19 

0.000 

0.263 

0.368 

0.211 

0.158 

3.26 

S2  Q6 

19 

0.000 

0.316 

0.368 

0.263 

0.053 

3.05 

S2  Q7 

19 

0.000 

0.368 

0.211 

0.421 

0.000 

3.05 

S2  Q8 

19 

0.000 

0.421 

0.474 

0.053 

0.053 

2.74 

S2Q13 

19 

0.000 

0.421 

0.421 

0.158 

0.000 

2.74 
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when  parametrics  should  be  used  came  from  the  Non-PCEI  Government  group  but  was 
only  42  percent  in  favor  of  that  proposition.  The  other  groups  range  from  0  to  20  percent. 
Again  the  impact  of  CER’s  being  a  type  of  parametric  estimate  could  be  significant. 

Items  S2Q3,  S2Q8,  and  S2Q13  concern  acquisition  phase.  Results  on  S2Q3, 
parametrics  are  more  appropriate  during  the  concept  development  phase,  are  mixed. 

PCEI  Industry  disagreed  with  that  item  44  percent  of  the  time,  PCEI  Government  73 
percent  of  the  time,  Non-PCEI  Industry  40  percent,  and  Non-PCEI  Government  26 
percent.  Results  for  S2Q8  and  S2Q13  provide  a  more  discernible  pattern.  Except  for 
Non-PCEI  Government,  there  is  70  to  90  percent  disagreement  with  limiting  the  use  of 
parametrics  to  early  phase  estimates  and  not  using  it  in  production  situations.  Non-PCEI 
Government  only  had  42  percent  disagreement  with  these  limitations. 

Item  S2Q6  concerns  whether  information  availability  should  eliminate  the  use 
parametrics.  The  PCEI  Industry  and  Government  groups  had  80  and  73  percent 
disagreement  with  that  premise,  respectively,  while  the  non-PCEI  groups  had  50  and  32 
percent  disagreement  with  that  premise. 

Overall  utility  is  defined  as  a  combination  all  seven  of  the  items  just  discussed,  each 
addressing  some  aspect  of  perceived  utility  such  as  use  during  any  acquisition  phase  or 
contract  type,  and  desire  to  expand  the  use  of  parametrics  (reference  Chapter  III  of  this 
thesis).  A  mean  response  is  developed  for  each  of  the  seven  items.  The  overall  perceived 
utility  measure  is  therefore  established  as  the  mean  of  the  seven  mean  responses  to  the 
seven  items  just  discussed.  A  comparison  of  the  mean  scores  for  the  overall  utility,  or 
usefulness  of  parametrics  is  shown  in  Figure  4. 


56 


This  measure  of  utility  is  more  a  measure  of  flexibility  becasue  the  items  that 
compose  it  question  whether  parametrics  should  be  use  in  any  situation.  The  results 


Figure  4.  Utility  of  Parametric  Estimating 


show  that  PCEI  Industry  and  Government  have  the  highest  belief  in  the  utility/flexibility 
of  parametrics,  Non-PCEI  Government  the  lowest  with  Non-PCEI  Industry  splitting  that 
difference. 

Investigative  Question  Five:  Is  there  a  difference  in  the  perceived  accuracy  and 
usefulness  of  the  six  types  of  parametric  applications?  The  intent  of  investigative 
question  five  is  to  evaluate  the  perceptions  of  accuracy  and  utility  of  the  different  types  of 
parametric  applications  across  the  different  acquisition  groups.  Section  3  of  the 
questionnaire  was  used  to  evaluate  differences  in  perceptions.  The  six  applications 
previously  identified  were:  1)  CERs  that  are  cost-to-cost,  2)  CERs  that  are  cost-to-non- 
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cost,  3)  commercial  models  that  estimate  the  entire  cost  of  a  proposal,  4)  commercial 
models  that  estimate  only  significant  portions  of  a  proposal,  5)  in-house  models  that 
estimate  only  significant  portions  of  a  proposal,  and  6)  in-house  models  that  estimate  the 
entire  cost  of  a  proposal.  Question  5  for  each  application,  provide  benefit  in  accuracy, 
was  used  to  evaluate  perceptions  of  accuracy  for  each  application.  Questions  1  through  4 
and  6  for  each  application,  were  used  to  evaluate  the  usefulness  or  utility.  These 
questions  are:  easy  to  develop  (or  learn),  easy  to  use,  frequently  used,  provide  benefit  in 
efficiency,  and  should  be  used  at  any  phase  of  the  acquisition.  Table  7  provides  the 
results  for  the  accuracy  application  questions. 

At  least  60  percent  of  the  PCEI  Industry  respondents  indicated  that  parametric 
estimating  techniques  provide  benefits  in  accuracy  regardless  of  the  application.  This 
view  is  generally  different  from  the  other  three  acquisition  groups.  For  the  other 
acquisition  groups,  perceptions  of  accuracy  were  lower  and  there  was  generally  a  drop-off 
in  perceived  accuracy  after  the  two  CER  categories.  The  exception,  Non-PCEI  Industry, 
had  a  75  percent  agreement  that  cost-to-cost  CERs  provide  a  benefit  in  accuracy,  but 
dropped  off  to  50  percent  for  cost-to-non-cost  CERs.  Non-PCEI  Industry  also  indicated  a 
56  percent  agreement  with  the  accuracy  benefit  of  commercial  models  used  to  estimate 
the  entire  cost  of  a  proposal,  but  less  than  40  percent  agreement  on  the  other  parametric 
types.  The  lower  agreement  level  and  drop-off  pattern  are  readily  discernible  when 
looking  at  the  two  government  groups.  PCEI  Government  respondents  provided  only  55 
percent  agreement  with  the  accuracy  benefit  of  cost-to-cost  CERs,  45  percent  agreement 
with  cost-to-non-cost  CERS,  falling  off  to  17  to  31  percent  for  the  other  parametric 
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applications.  A  majority  of  the  PCEI  government  group  provided  neutral  responses 
concerning  accuracy  benefits  obtained  from  the  other  four  applications.  The  Non-PCEI 


Table  7.  Responses  to  Questionnaire  Items  for  Investigative  Question  Five 


Strongly 

Strongly 

Item 

n 

Disagree  Disagree 

Neither 

Agree 

Agree 

Mean 

PCEI  Industry 

CERS  Cost  to  Cost 

31 

0.000 

0.129 

0.194 

0.516 

0.161 

3.71 

CERS  Cost  to  NonCost 

32 

0.000 

0.094 

0.250 

0.469 

0.188 

3.75 

Commercial  Models  -  entire 

33 

0.030 

0.030 

0.333 

0.455 

0.152 

3.67 

Commercial  Models  -  portion 

30 

0.000 

0.033 

0.233 

0.533 

0.200 

3.90 

In-house  Models  -  portion 

33 

0.000 

0.030 

0.333 

0.485 

0.152 

3.76 

In-house  Models  -  entire 

32 

0.000 

0.031 

0.375 

0.406 

0.188 

3.75 

PCEI  Government 

CERS  Cost  to  Cost 

18 

0.000 

0.056 

0.389 

0.444 

0.111 

3.71 

CERS  Cost  to  NonCost 

18 

0.056 

0.056 

0.444 

0.389 

0.056 

3.41 

Commercial  Models  -  entire 

17 

0.000 

0.176 

0.647 

0.118 

0.059 

3.13 

Commercial  Models  -  portion 

17 

0.000 

0.118 

0.647 

0.176 

0.059 

3.25 

In-house  Models  -  portion 

16 

0.000 

0.125 

0.563 

0.250 

0.063 

3.33 

In-house  Models  -  entire 

16 

0.000 

0.063 

0.688 

0.188 

0.063 

3.33 

Non-PCEI  Industry 

CERS  Cost  to  Cost 

8 

0.000 

0.125 

0.125 

0.500 

0.250 

3.88 

CERS  Cost  to  NonCost 

8 

0.000 

0.500 

0.000 

0.375 

0.125 

3.13 

Commercial  Models  -  entire 

9 

0.000 

0.111 

0.333 

0.556 

0.000 

3.44 

Commercial  Models  -  portion 

8 

0.000 

0.125 

0.625 

0.250 

0.000 

3.13 

In-house  Models  -  portion 

9 

0.000 

0.111 

0.556 

0.333 

0.000 

3.22 

In-house  Models  -  entire 

8 

0.000 

0.375 

0.250 

0.375 

0.000 

3.00 

Non-PCEI  Government 

CERS  Cost  to  Cost 

19 

0.000 

0.211 

0.368 

0.316 

0.105 

3.32 

CERS  Cost  to  NonCost 

19 

0.000 

0.158 

0.526 

0.263 

0.053 

3.21 

Commercial  Models  -  entire 

19 

0.158 

0.158 

0.579 

0.105 

0.000 

2.63 

Commercial  Models  -  portion 

19 

0.053 

0.368 

0.474 

0.105 

0.000 

2.63 

In-house  Models  -  portion 

19 

0.053 

0.263 

0.579 

0.105 

0.000 

2.74 

In-house  Models  -  entire 

19 

0.053 

0.368 

0.474 

0.105 

0.000 

2.63 

Government  responses  display  a  substantially  lower  level  with  agreement  of  only  1 1 
percent  agreeing  with  accuracy  benefit  of  the  four  types  of  applications  other  than  CERs. 

The  associated  histogram  of  the  means  scores  for  the  perceptions  of  accuracy  by 
parametric  application  is  shown  in  figure  5.  For  the  PCEI  Industry  group,  there  is  a 
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PERCEIVED  ACCURACY 


Figure  5.  Perceived  Accuracy  by  Parametric  Application 


uniform  perception  of  accuracy  among  the  different  parametric  applications.  This 
uniform  perception  of  accuracy  does  not  exist  with  the  other  acquisition  groups.  The 
other  groups  tend  to  show  a  reduction  in  perceived  accuracy  as  you  move  away  from 
simple  CERS  to  more  complex  parametric  applications  such  as  commercial  and  in-house 
models.  The  reduction  in  perceived  accuracy  of  CERs  and  models  is  most  noticeable  for 
the  Non-PCEI  Government  group. 

Overall,  the  data  suggests  that  PCEI  Industry  group  sees  very  little  difference  in 
accuracy  between  the  six  types  of  parametric  applications,  whereas,  the  other  three 
groups  perceive  CERs,  particularly  cost-to-cost,  to  be  more  accurate  than  the  other  types 
of  parametric  applications.  The  most  noticeable  difference  in  perception  is  from  the  Non- 
PCEI  Government  group. 
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Investigative  question  five  also  addresses  differences  in  perceived  utility  between 
the  different  parametric  applications.  Data  from  Section  3  of  the  survey  was  used  to 
evaluate  utility  in  five  components:  ease  to  develop,  ease  to  use,  frequency  of  use, 
benefit  in  efficiency,  and  can  be  used  at  any  acquisition  phase.  Since  commercial  models 
are  already  developed,  the  item  was  changed  to  ease  to  learn  on  the  two  commercial 


Table  8.  Responses  to  Questionnaire  Items  for  Investigative  Question  Five 


Summary  of  Mean  Responses  for  Different  Application  Types 


Ease  to  Ease  to  Frequently 

Develop  Use  Used 

Efficiency  Used  Average 

Benefit  Anytime  (Utility) 

Cost-to-Cost  CER's 

PCEI-Industry 

3.74 

4.28 

3.81 

4.29 

3.81 

3.99 

PCEI-Govermnent 

3.59 

3.94 

3.59 

4.18 

3.76 

3.81 

Non-PCEI  -Industry 

4.00 

4.38 

4.00 

4.25 

3.75 

4.08 

Non-PCEI  -Government 

3.58 

3.89 

3.95 

3.79 

3.47 

3.74 

Cost-to-Non-Cost  CER's 

PCEI-Industry 

3.23 

4.06 

3.55 

4.10 

3.87 

3.76 

PCEI-Govemment 

2.71 

3.41 

3.06 

3.65 

3.47 

3.26 

Non-PCEI  -Industry 

3.75 

4.25 

3.88 

4.13 

3.63 

3.93 

Non-PCEI  -Government 

3.26 

3.79 

3.74 

3.74 

3.21 

3.55 

Commercial 

Models  for  Entire  Proposals 

PCEI-Industry 

2.75 

2.84 

3.09 

3.69 

3.72 

3.22 

PCEI-Govemment 

2.69 

2.94 

2.50 

3.44 

3.25 

2.96 

Non-PCEI  -Industry 

2.67 

3.00 

3.00 

3.67 

3.33 

3.13 

Non-PCEI  -Government 

2.37 

2.53 

2.37 

3.32 

2.68 

2.65 

Commercial  Models  for  Portions  of  a  Proposal 

PCEI-Industry 

2.97 

3.23 

3.43 

4.03 

3.83 

3.50 

PCEI-Govemment 

3.00 

3.13 

3.06 

3.69 

3.44 

3.26 

Non-PCEI  -Industry 

2.63 

3.25 

3.13 

3.25 

3.13 

3.08 

Non-PCEI  -Government 

2.47 

2.58 

2.95 

3.37 

2.68 

2.81 

In-House  Models  for  Portions  of  a  Proposals 

PCEI-Industry 

2.00 

3.03 

3.13 

3.81 

3.47 

3.09 

PCEI-Govemment 

2.47 

3.00 

2.93 

3.73 

3.33 

3.09 

Non-PCEI  -Industry 

3.22 

3.22 

2.78 

3.56 

2.89 

3.13 

Non-PCEI  -Government 

2.84 

2.79 

3.00 

3.42 

2.84 

2.98 

In-House  Models  for  Entire  Proposals 

PCEI-Industry 

1.74 

2.61 

2.58 

3.90 

3.61 

2.89 

PCEI-Govemment 

2.60 

2.93 

3.00 

3.47 

3.27 

3.05 

Non-PCEI  -Industry 

2.38 

3.00 

2.63 

3.50 

3.25 

2.95 

Non-PCEI  -Government 

2.63 

2.63 

2.53 

3.26 

2.74 

2.76 
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model  applications.  The  mean  response  for  the  five  components  develops  a  composite 
utility  measure.  Mean  responses  for  each  acquisition  group  and  each  type  application 
were  calculated  along  with  an  overall  utility  response.  Table  8  provides  the  results  for  the 
utility  application  questions. 

The  associated  histogram  of  the  means  scores  for  the  perceptions  of  overall  utility 
by  parametric  application  is  shown  in  figure  6. 
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PCEI  Industry  Non-PCEI  Industry  Non-PCEI  Govt 


Figure  6.  Perceived  Utility  by  Parametric  Application 

The  general  pattern  is  similar  to  that  of  the  accuracy  histogram.  There  is  a  reduction 
in  the  perception  of  utility  for  commercial  and  in-house  parametric  models  as  compared 
to  CER's.  However,  unlike  accuracy,  there  is  also  a  drop-off  in  PCEI  Industry’s 
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perception  of  utility  for  models  that  estimate  the  entire  cost  of  a  proposal.  This  drop-off 
pattern  is  more  pronounced  for  the  utility  construct  than  accuracy.  The  reduction  in 
perceived  utility  between  CERs  and  models  is  most  noticeable  for  Non-PCEI 
Government.  It  is  also  noteworthy  that  the  Non-PCEI  Government  perception  of  utility 
of  CER's  is  comparable  with  the  other  acquisition  groups.  It  is  also  observable  that  the 
Non-PCEI  Government  perception  of  utility  of  models  is  somewhat  lower  than  the  other 
acquisition  groups.  The  Non-PCEI  Industry  drop-off  is  also  very  noticeable. 

As  previously  discussed,  the  utility  construct  is  composed  of  five  different  items.  The 
histograms  for  the  mean  responses  of  each  of  the  utility  items  are  shown  in  figures  7 
through  1 1 . 


EASE  OF  USE 


Figure  7.  Perceived  Ease  of  Use  by  Parametric  Application 
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Mein  Response 


Frequency  of  Use 
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Figure  8.  Perceived  Frequency  of  Use  by  Parametric  Application 
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Figure  9.  Perceived  Efficiency  Benefit  by  Parametric  Application 
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Use  During  Any  Acqusition  Phase 
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Figure  10.  Perceived  Use  During  Any  Acquisition  Phase  by  Parametric  Application 
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The  histograms  for  the  five  items  follow  a  similar  pattern  as  the  composite  utility 
measure  -  the  mean  response  drop-off  from  CERs  to  parametric  models.  This  drop-off  is 
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most  pronounced  for  Non-PCEI  Government  in  nearly  every  case.  The  drop-off  is 
noticeable  for  Non-PCEI  Industry  in  nearly  every  case.  PCEI  Industry  had  a  drop-off  in 
ease  of  use,  frequency  of  use,  and  ease  to  develop/leam,  but  little  if  any  drop-off  in 
efficiency  benefit  and  use  during  any  acquisition  phase.  PCEI  Government  had  a  drop¬ 
off  from  the  cost-to-cost  CER’s  only.  Just  like  accuracy,  PCEI  Government  tended  to 
view  cost-to-non-cost  CER’s  on  a  similar  level  with  models. 

The  data  indicates  all  groups  have  some  variance  in  their  perceived  utility  of 
parametrics  applications.  The  PCEI  Industry  group  sees  a  decrease  in  the  utility  of 
parametrics  when  using  commercial  or  in-house  models  to  estimate  entire  proposals. 
However,  if  estimating  only  a  portion  of  the  proposal,  these  models  are  perceived  to  be  as 
useful  as  CERs.  The  other  three  acquisition  groups  perceive  a  decrease  in  the  usefulness 
of  commercial  and  in-house  models  when  compared  with  CERs,  though  sometimes  only 


cost-to-cost  CERs. 


V.  Conclusions  and  Recommendations 


Overview 

The  previous  chapter  presented  and  discussed  the  results  of  our  research  study.  This 
chapter  concludes  the  discussion  by  providing  possible  ramifications  and  larger 
implications  of  the  data  evaluated  against  each  research  question.  Explanatory 
information  is  discussed  with  regard  to  each  investigative  question  presented  in 
Chapter  III.  The  five  research  questions  are: 

1.  Has  the  use  of  parametric  estimating  increased  and  is  more  effort  needed  to 
expand  its  implementation? 

2.  What  are  some  of  the  perceived  barriers  to  implementation? 

3.  Do  different  acquisition  groups  have  different  perceptions  on  the  accuracy  of 
parametrics? 

4.  How  useful  is  parametric  estimating  to  government  contracting  as  influenced  by 
phase  of  the  acquisition  cycle,  contract  type,  and  information  availability? 

5.  Is  there  a  difference  in  the  perceived  accuracy  and  usefulness  of  the  six  types  of 
parametric  applications? 

Summary  Findings.  Management  Implications  and  Recommendations  for  Research 
Question  One 

Has  the  use  of  parametric  estimating  increased  and  is  more  effort  needed  to  increase 
its  use? 

Overall,  all  acquisition  groups  perceived  an  increase  in  the  use  of  parametric 
estimating,  and  all  groups,  except  Non-PCEI  Government  felt  more  effort  was  needed  to 
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expand  the  use  of  parametrics.  The  responses  were  only  slightly  above  neutral  (neither 
agree  or  disagree)  suggesting  that  parametric  expansion  may  not  have  increased 
significantly  or  as  much  as  desired. 

This  information  provides  two  management  implications.  First,  we  question  why  is 
there  not  a  strong  endorsement  that  the  use  of  parametric  estimating  has  increased.  There 
are  many  success  stories  being  shared  but  apparently,  not  many  real  uses  in  contracting 
situations.  Ways  need  to  be  found  to  bridge  that  gap.  The  second  implication  is  to 
determine  the  cause  for  why  Non-PCEI  Government  people  do  not  share  the  same  level 
of  endorsement  of  the  proposition  that  more  effort  is  needed  to  expand  its  use.  One 
possible  answer  is  that  Non-PCEI  Government  people  may  not  be  party  to  the  success 
events  advocated  by  the  PCEI  and  therefore,  cannot  justify  expending  additional  effort  in 
expanding  its  use.  This  is  evidenced  by  the  first  management  implication. 

Recommendations.  The  PCEI  needs  to  share  successes  with  Non-PCEI  government 
people  to  gain  greater  endorsement  from  them  for  applying  more  effort  to  expansion  of 
parametric  applications. 

Summary  Findings,  Management  Implications  and  Recommendations  for  Research 
Question  Two 

What  are  some  of  the  perceived  barriers  to  implementation? 

Data,  training,  and  model  development  were  ranked  high  as  barriers  by  all 
acquisition  groups.  TINA  issues  and  lack  of  government  support  were  ranked  high  by 
Non-PCEI  Industry.  Non-PCEI  Government  ranked  utility  and  accuracy  concerns  highly. 
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The  most  significant  management  implication  concerns  data.  The  retrieval  and 
organization  of  historical  data  is  a  precursor  to  any  parametric  expansion  effort.  The 
investment  to  perform  that  task  should  be  placed  as  a  higher  priority  than  it  has  been  in 
the  past.  The  President  of  Lockheed  Martin  Tactical  Aircraft  Systems  has  stated 
parametric  “models  must  be  supported  by  current  databases,  and  it  is  not  an  easy 
endeavor  to  develop  and  maintain  these  databases”  (Hancock,  1996:2).  This  reflects  the 
real  concern  a  contractor  encounters  when  trying  to  develop  cost-to-non-cost  CERs  and 
in-house  models. 

Some  insight  on  perceived  barriers  was  provided  from  the  March  1998  parametric 
conference.  Mr.  Joe  Lewis  of  Boeing,  a  keynote  speaker  at  the  conference,  stated  data  is 
the  major  obstacle  to  overcome  in  expanding  the  use  of  parametric  estimating.  He 
estimated  that  nearly  fifteen  man-years  had  been  invested  in  reviewing  and  correlating  the 
historical  database  information  for  Boeing’s  parametric  effort  (Lewis,  1998).  GE  has 
invested  several  million  in  the  development  of  its  COMPEAT  model  with  data  retrieval  a 
large  part  of  that  investment. 

During  the  1998  conference,  a  question  was  asked  why  old  data  is  important.  In 
response,  it  was  stated  both  old  data  and  new  data  are  important.  Old  data  helps  to 
establish  the  impact  of  technological  advancements  by  looking  at  trends  and  what 
happens  to  cost  when  efforts  are  made  to  advance  technology.  New  data  is  significant 
because  it  is  easier  to  correlate,  thereby  providing  a  current  point  of  projection 
(Parametric  Cost  Estimating  Joint  Industry/Govemment  Initiative  Workshop,  1998). 
Boeing  believes  you  can  not  proceed  on  parametrics  until  you  know  where  you  have 
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been.  In  contracting,  it  does  not  appear  that  most  contractors  acknowledge  this  and  are 
ready  to  make  the  required  investment. 

During  the  1998  conference,  Mr.  H.  Apgar,  MCR,  stated  that  in  his  consulting  role, 
mock  audits  are  often  run  on  contractor’s  attempts  to  establish  parametric  systems.  The 
contractors  are  questioned  about  what  historical  data  is  available,  or  what  plans  exist  to 
retrieve,  organize  and  normalize  data.  The  answer  is  invariably,  none  (Parametric  Cost 
Estimating  Joint  Industry/Govemment  Initiative  Workshop,  1998).  This  condition  is  not 
supportive  of  institutionalizing  parametric  estimating.  Data  is  needed  not  only  to  develop 
CERs  but  also  to  validate  and  calibrate  commercial  models.  Without  data,  there  is  no 
expansion  of  parametric  estimating. 

Mr.  Apgar  was  one  of  the  founders  of  ISPA  twenty  years  ago.  He  tells  the  story  of 
how  he  expected  it  to  take  a  few  years  to  change  the  world.  Twenty  years  have  since 
passed,  and  it  now  appears  the  world  is  ready  for  change,  but  the  data  to  change  the  world 
is  not  available.  Expectations  are  unusually  high  when  a  quicker  or  better  way  of  doing 
something  is  announced.  Most  people  expect  immediate  results.  Many  companies 
expect  two  and  three  year  paybacks  on  savings  programs.  It  has  been  pointed  out  that  the 
benefits  to  be  derived  from  parametric  estimating  cannot  be  realized  until  the  up-front 
cost  to  calibrate  and  validate  the  models  has  been  invested  and  until  the  models  are 
understood  and  accepted  as  a  reliable  predictor  of  cost  (Allocation  of  Resources,  1996:8) 
This  can  only  occur  with  good  data. 

Research  directed  to  this  end  may  help  to  improve  parametric  estimating.  Model 
validation  is  not  feasible  until  a  contractor  can  retrieve,  organize,  and  correlate  its 
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historical  data.  The  PCEI  would  do  well  to  stress  the  importance  of  this  concept. 

Because  of  the  volume  and  variety  of  data  needed,  it  is  a  struggle  to  accumulate  and 
accurately  manipulate  required  historical  data  to  make  models  credible.  If  there  is  a 
single  reason  why  parametric  estimates  may  take  some  time  to  implement,  it  is  likely  the 
data  issue.  Estimates  for  how  long  it  will  take  to  institutionalize  parametric  estimating 
vary  from  2  to  15  years  (Parametric  Cost  Estimating  Joint  Industry/Govemment  Initiative 
Workshop,  1998). 

Another  implication  from  the  barriers  information  addresses  the  need  for 
government  approval  authorities  to  provide  a  firm  mechanism  for  dealing  with 
subcontract  cost  or  pricing  data  when  using  parametric  estimating.  This  was  the  number 
one  concern  for  Non-PCEI  Industry  and  it  is  a  valid  concern.  In  one  specific  case,  an 
industry  executive  stated  that  it  is  too  time  consuming  to  get  subcontractor  proposals  for 
each  follow-on  procurement.  Parametric  estimating  was  a  more  efficient  way  of 
accomplishing  this  (Fowler,  1997).  Many  acknowledge  that  subcontract  cost  and  pricing 
data  are  the  major  impediments  in  reaching  negotiated  settlements.  There  are  few  if  any 
examples  of  buying  offices  accepting  parametrically  estimated  subcontract  cost  in  lieu  of 
subcontract  cost  and  pricing  data. 

There  is  also  a  need,  especially  on  the  government  side,  to  be  aware  of  and  prepare 
for  required  training.  The  PCEI  identified  training  as  an  important  focus  area  from  the 
very  beginning.  Courses  have  been  and  are  still  being  developed  to  provide  general  and 
specialized  knowledge  in  statistical  mathematics  and  parametric  model  use  (Working 
Relationships,  1997:16).  Commercial  model  vendors  provide  training  on  the  use  of  their 
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models  but  it  is  generally  acknowledged  that  it  takes  a  lot  of  on-the-job  use  to  gain 
expertise  with  most  commercial  models.  Some  contractors  do  or  will  have  to  provide 
specific  training  on  applicable  contractor  models. 

There  is  some  uncertainty  about  what  exactly  should  be  taught  for  specific 
situations.  A  typical  parametrician  is  certainly  knowledgeable  in  graduate  level  statistics, 
but  the  average  buyer/contracting  officer  is  not  going  to  be  a  parametrician.  For  buying 
offices,  the  problem  may  not  be  too  pronounced.  If  a  contractor  is  using  CERs,  the 
statistical  knowledge  is  needed  during  its  development,  not  during  its  use.  Expanded  use 
of  simple  CERs,  should  be  business  as  usual.  DCMC  and  DCAA  personnel  have  a 
greater  need  for  in-depth  statistical  knowledge  as  they  need  to  ascertain  the  predicative 
accuracy  of  whatever  parametric  application  is  being  proposed  for  use.  The  amount  of 
statistical  training  needed  at  product  divisions  is  more  a  function  of  how  involved  the 
product  divisions  want  to  be  during  the  development  of  parametric  applications.  In  most 
cases,  that  current  involvement  is  low.  That  is  not  necessarily  bad  or  unusual  since  CERs 
can  be  thought  of  as  rates.  Once  the  DCMC  or  DCAA  recommends  acceptance  of  the 
rates,  the  buying  offices  seldom  challenge  these  recommendations.  There  is  no  reason 
why  this  change  should  be  any  different. 

Training  on  use  of  specific  parametric  models  is  a  different  case.  Most  models  are 
fairly  complex  and  specific  training  needs  must  be  identified  on  a  case  by  case  basis.  The 
importance  of  training  was  greatly  emphasized  at  the  March  1998  conference. 

Recommendations.  General  procedures  need  to  be  established  for  kinds  of  data  that 
are  needed,  and  for  cost  effective  ways  of  retrieving  and  organizing  data.  Some 
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suggestions  have  proposed  ideas  such  as  bringing  in  retirees  who  have  first  hand 
knowledge  of  the  historical  data.  In  addition,  a  panel  of  working  level  contracting 
officers,  price  analyst,  and  reviewers  should  be  convened  to  provide  recommendations  on 
how  buying  offices  should  address  the  apparent  conflict  that  exists  between  FAR  15 
requirements  and  parametric  estimates  as  it  relates  to  subcontract  pricing.  Concerning, 
training,  the  only  real  action  suggested  is  for  buying  activities  to  be  aware  of  any  future 
receipts  of  parametrical  model  estimated  proposals  to  allow  sufficient  lead-time  to 
accomplish  the  necessary  training. 

Summary  Findings,  Management  Implications  and  Recommendations  for  Research 
Question  Three 

Do  different  acquisition  groups  have  different  perceptions  on  the  accuracy  of 
parametrics? 

PCEI  Industry  and  Non-PCEI  Government  have  vastly  opposing  perceptions  on  the 
accuracy  of  parametric  estimates,  with  the  other  two  groups  about  halfway  between  the 
high/low  extremes.  Some  suspect  the  problem  stems  from  lack  of  involvement. 
Approximately  250  people  attended  the  March  1998  parametric  conference,  roughly  half 
government  and  half  industry.  Far  less  than  twenty  were  from  buying  activities.  A  belief 
exists  that  buying  activities  are  reluctant  to  accept  parametric  estimates.  DCMC  and 
DCAA  have  been  involved  in  the  PCEI  for  nearly  four  years  while  the  applications  were 
being  developed.  Buying  activity  involvement  has  been  slim  to  none.  The  PCEI 
considers  the  contractor  to  be  responsible  for  integrating  the  buying  activity  downstream 
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once  the  parametric  applications  are  defined.  There  is  no  consensus  on  the  strengths  or 
deficiencies  associated  with  lack  of  early  buying  activity  involvement,  only  concern  over 
the  lack  of  acceptance. 

The  magnitude  of  the  problem  concerning  customer  acceptance  is  a  function  of  the 
type  of  application  used.  The  greater  the  complexity  of  the  parametric  application,  the 
more  likely  customer  acceptance  will  be  a  problem.  Do  buying  functions  need  to  be 
involved  in  establishing  procedures  for  the  proper  use  of  parametric  models?  How  long 
will  it  take  to  be  able  to  correctly  use  the  models?  How  much  involvement  will  the 
buying  office  need  in  the  process  of  model  validation  and  calibration?  This  is  where 
confidence  in  the  parametric  estimate  is  gained.  The  buying  activity  needs  to  become 
involved  sooner  if  the  applications  used  are  more  complex  because  the  application 
defines  the  complexity  of  the  knowledge  necessary  to  have  confidence  in  the  quality  of 
the  bidding  methodology. 

Addressing  the  question  of  accuracy  is  more  a  question  of  addressing  the  possible 
inverse  relationship  between  accuracy  and  efficiency  of  preparation.  The  initiative’s 
goals  clearly  include  using  parametrics  to  develop  estimates  at  a  lower  cost  with  similar 
accuracy.  Mr.  Jim  Collins,  Co-Chairman  of  the  PCEI,  stated  the  benefits  of  parametric 
cost  estimating  techniques  include  reduction  in  proposal  preparation  cost,  reduction  in 
proposal  cycle  time,  and  more  reliable  estimates.  Collins  also  states  that  without 
demonstrated  benefits  and  reliable  estimates,  there  will  be  no  reason  for  change  and 
parametric  cost  estimating  processes  will  not  be  adopted  (Collins,  1996:8).  The  Manager 
of  Military  Proposals  at  GE  Aircraft  Company  stated: 
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There  are  no  required  or  suggested  levels  of  precision  for  any  cost 
estimating  technique  as  previously  referenced.  The  use  of  all  cost 
estimating  systems  requires  that  it  form  an  acceptable  basis  for 
negotiations.  The  contractors  use  of  a  cost  model  should  not  be  subject  to 
a  different  set  of  rules  or  guidelines  than  their  other  cost  estimating 
routines.  The  level  of  precision  would  then  be  reasonable  to  apply  at  a 
total  proposal  price  since  this  is  the  basis  for  negotiation.  (Brandstetter, 

1996:6) 

Another  industry  executive  stated  precision  is  a  matter  of  judgement,  higher  levels  of 
precision  are  needed  on  fixed  priced  contracts  (Calibration/Validation,  1996:1 1).  The 
mention  of  contract  type  and  precision  of  estimate  is  interesting.  In  her  January  1997 
letter,  the  Principal  Deputy  Assistant  Secretary  (Acquisition  &  Management)  stated 
parametrics  “...can  produce  contract  price  estimates  of  equal  or  better  reliability  that 
traditional  bottom-ups  estimates”  (Druyun,  1997:1).  The  basis  of  these  equal  accuracy 
statements  contradicts  the  status  quo  and  any  empirical  or  other  evidence  presented  in  the 
literature. 

The  Parametric  Cost  Estimating  Initiative  does  not  ignore  this.  The  PCEI  Steering 
Committee  stated  the  degree  of  accuracy  required  should  be  as  accurate  as  the  current 
proposal  method  in  predicting  cost.  However,  the  “principal  goal  of  the  initiative  is  to 
lower  estimating  and  oversight  costs.. .to  mandate  a  specific  [accuracy]  percentage  would 
be  counterproductive.  One  would  hope  be  able  to  predict  cost  within  +10%  for 
established  programs  but  accuracy  within  ±25%  may  be  all  that  is  achievable  on  a 
development  contract"  (Collins  &  Eck,  1997a).  The  acceptability  of  those  target 
percentages  is  uncertain.  Will  contracting  officers  subscribe  to  estimating  techniques 
with  ±10%  accuracy  on  established  programs?  Several  questions  concerning  accuracy 
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were  presented  in  the  Parametric  Cost  Estimating  Initiative  newsletters.  How  will  the 
contractor  demonstrate  the  model  is  reasonably  accurate?  What  are  the  criteria  for 
measuring  accuracy?  What  is  a  reasonable  variance  percentage?  The  response  was  the 
contractor  must  have  a  procedure  to  demonstrate  accuracy  to  provide  credible  estimates. 
Accuracy  must  be  similiar  to  existing  methods  (Calibration/Validation,  1996:11  -  12). 

The  accuracy  question  is  a  tough  one  to  address  and  the  PCEI  has  left  many 
questions  unanswered  in  dealing  with  it.  There  has  yet  to  be  a  detailed  documented 
discussion  of  accuracy  topics  such  as  sources  of  inaccuracy  and  quantifying  inaccuracy 
for  risk  analysis  purposes.  The  Parametric  Cost  Estimating  Newsletter  does  contain 
numerous  examples  of  calibration  efforts  at  different  sites.  Calibration  is  the  process  of 
adjusting  a  commercially  developed  model  from  industry  averages  to  customized  local 
experience  and  performance.  However,  accuracy  must  contend  with  the  reality  of 
substantial  non-recurring  start-up  cost.  Given  a  fixed  level  of  information,  time,  and 
resources,  how  can  parametrically  oriented  estimating  approaches  improve  upon  what  is 
done  now?  Little  evidence  exists  that  adequately  addresses  the  accuracy  efficiency  trade¬ 
off. 

For  management,  this  implies  without  customer  belief  that  the  estimating  technique 
used  is  accurate,  it  will  not  be  accepted  and  used.  The  discussion  of  accuracy  perceptions 
becomes  clearer  when  different  parametric  applications  are  considered  in  investigative 
question  five. 
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Summary  Findings.  Management  Implications  and  Recommendations  for  Research 
Question  Four 

How  useful  is  parametric  estimating  to  government  contracting  as  influenced  by 
phase  of  acquisition  cycle,  contract  type,  and  information  availability? 

Both  the  PCEI  Industry  and  Government  groups  responded  more  positively  to  the 
use  of  parametric  applications  in  any  situation  as  compared  with  the  Non-PCEI 
Government  group.  The  Non-PCEI  Government  group  did  not  support  limiting 
parametrics  to  cost  type  contracts,  though  this  answer  will  be  later  contradicted  by 
findings  in  investigative  question  five. 

This  information  implies  that  those  disposed  to  prepare  and  negotiate  proposals, 
both  Non-PCEI  Government  and  Industry,  are  somewhat  unsure  about  using  parametrics 
in  situations  where  it  may  not  have  been  previously  used.  While  traditional  views  may 
advocate  that  there  are  situational  limitations  to  the  use  of  parametrics,  the  PCEI,  both 
government  and  industry,  believe  in  the  unlimited  use  of  parametrics  in  all  situations.  It 
appears  that  more  demonstration  of  the  flexibility  and  benefits  of  using  specific 
techniques  will  have  to  be  sold  to  the  primary  users. 

One  of  the  senior  service  acquisition  officials  provided  another  perspective  of 
buying  activity  attitudes  at  the  March  conference.  He  stated  it  is  unquestionable  the 
acquisition  side  of  the  house  at  the  higher  management  levels  categorically  supports  this 
initiative  (Parametric  Cost  Estimating  Joint  Industry/Govemment  Initiative  Workshop, 
1998).  All  the  service  executives  emphasized  that  more  parametric  estimation  is 
necessary  because  DOD  can  no  longer  man  to  the  levels  necessary  to  do  things  the  way 
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they  have  always  been  done.  The  major  issue  is  credibility;  “there  is  a  distinction 
between  something  that  is  theoretically  correct  and  something  that  is  practically  useful” 
(Rany,  1997).  Discussions  of  parametric  estimating  utility,  just  like  accuracy,  are 
difficult,  if  not  impossible.  One  must  determine  what  type  of  parametric  application  is 
being  discussed.  These  issues  are  addressed  in  investigative  question  five. 

Summary  Findings,  Management  Implications  and  Recommendations  for  Research 
Question  Five 

Is  there  a  difference  in  the  perceived  accuracy  and  usefulness  of  the  six  types  of 
parametric  applications? 

As  parametrics  moves  from  the  simpler  CERs  to  the  more  complex  models,  only 
PCEI  Industry  did  not  experience  a  drop-off  in  the  perception  of  accuracy.  The  drop-off 
was  more  pronounced  after  cost-to-cost  CERs  and  more  pronounced  for  Non-PCEI 
Government.  There  was  some  degree  of  a  drop-off  in  the  perception  of  utility  in  all  four 
acquisition  groups.  PCEI  Industry,  perceived  noticably  less  over  all  utility  from 
parametric  models  that  estimate  entire  proposals.  For  both  Non-PCEI  groups,  the  overall 
dropoff  in  the  perception  of  utility  from  CERs  to  any  type  model  is  distinct.  PCEI 
Government  saw  utility  drop  off  after  cost-to-cost  CERs. 

The  overall  utility  construct  was  composed  of  five  items:  ease  to  develop  or  learn, 
ease  to  use,  frequency  of  use,  benefit  in  efficiency,  and  can  be  used  at  any  acquisition 
phase.  When  evaluating  ease  of  use,  frequency  of  use,  and  ease  to  develop  or  learn,  the 
drop-off  effect  was  noticeable  and  consistent  across  all  groups.  When  looking  at 
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efficiency  benefit  and  use  during  any  acquisition,  the  drop-off  was  less  pronounced  but 
did  exist,  more  for  Non-PCEI  Government  and  less  for  PCEI  Industry 

For  management,  the  differences  between  CERs  and  parametric  models  implies  we 
may  be  using  a  common  term  for  concepts  and  activities  that  are  different  in  several 
significant  aspects.  This  is  significant  for  several  reasons.  First,  it  indicates  that  PCEI 
efforts  to  expand  the  use  of  CERs  will  be  understood  and  supported  because  the  providers 
and  customers  already  believe  in  the  benefits  of  this  type  of  parametric  approach. 
Conversely,  PCEI  efforts  to  expand  the  use  of  parametric  models  must  demonstrate  the 
models  provide  at  least  as  much  utility  and  accuracy  as  current  methodologies. 

Discussing  parametrics  by  the  type  of  application  is  necessary  because  it  provides  a  basis 
for  activities  required  to  expand  the  use  of  different  types  of  applications. 

The  PCEI  objectives  and  philosophies  advocate  a  position  that  parametric 
applications  can  give  it  all  at  once,  quick,  cheap,  and  accurate.  Is  this  a  contradiction 
with  the  propositions  stated  in  the  accepted  literature?  Also,  is  this  adverse  to  the 
opinions  of  other  acquisition  groups,  particularly  Non-PCEI  government?  If  the  entire 
context  of  parametric  estimating  is  considered,  it  is  possible,  even  probable,  that  certain 
parametric  applications  can  give  you  quicker,  faster,  and  better  estimates  in  certain 
situations.  Others  will  take  longer,  cost  more,  and  be  less  accurate  than  traditional 
techniques.  The  Director  of  Defense  Procurement’s  comments  about  parametrics  being 
another  tool  could  be  right  on  the  money  so  the  question  becomes  when  do  we  use  what 
tool?  (Spector,  1995:1).  The  worst  thing  we  could  do  is  to  use  a  wrench  to  remove  a 
splinter. 
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The  general  term  parametric  estimating  is  very  broad.  It  is  analogous  to  a  farmer 
saying  we  are  planting  food  seeds.  Expanding  the  use  of  cost-to-cost  CERs  is  very 
different  from  expanding  the  use  of  commercial  models.  Different  from  both  are  the 
problems  of  developing  and  expanding  the  use  of  in-house  models.  Separate 
requirements  for  each  application  should  be  provided  for  training,  time  and  effort, 
limitations  and  barriers,  and  appropriate  situations  of  use.  Defining  the  issues  associated 
with  each  of  the  six  types  allows  a  cost  benefit  analysis  to  be  made  about  each  of  the 
specific  types.  This  should  improve  management  decision  making  concerning  the 
appropriate  use  of  parametric  applications. 

The  concept  of  different  parametric  applications  is  very  important  when  judging  the 
worth  of  the  PCEI.  It  is  worth  taking  a  look  at  the  accomplishments  of  the  PCEI  from  the 
perspective  of  the  different  types  of  parametric  applications.  Information  about  the 
accomplishments  of  PCEI  is  derived  from  two  sources,  the  Parametric  Cost  Estimating 
Initiative  newsletters  between  April  1996  and  December  1997  and  the  3-4  March  1998 
Parametric  Cost  Estimating  Joint  Industry/Govemment  Initiative  workshop.  The  real 
worth  of  the  PCEI  is  revealed  from  their  accomplishments. 

Lockheed  Martin  Tactical  Aircraft  System’s  (LMTAS)  efforts  to  estimate 
engineering  design  cost  based  on  hours  per  engineering  drawing  change  is  frequently 
mentioned  in  the  newsletters.  Since  this  is  a  cost-to-non-cost  CER,  the  data  had  to  come 
from  two  different  sources.  Correlating  and  validating  the  data  historical  database 
sources  was  difficult.  Boeing  Information  &  Communications  System’s  (BI&CS)  effort 
involved  establishing  a  process  for  developing,  applying  and  validating  CERs.  This 
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approach  involved  a  titanic  effort  to  analyze  historical  data  from  38  databases,  150 
contracts  spanning  20  years,  and  involved  numerous  programs  and  millions  of  hours. 

The  Boeing  effort  may  have  begun  as  early  as  1989,  prior  to  the  PCEI,  with  a  main  goal 
of  taking  most  of  the  judgment  out  of  their  proposal  estimates.  Boeing  claims  the  result 
could  achieve  more  than  a  “50%  reduction  in  proposal  flowtimes”  (Executive  Summary, 
1997:1 1).  There  are  several  other  Lab  Site  accomplishments  like  those  at  LMTAS  and 
Boeing.  In  all  the  cases,  the  efforts  involve  going  back  and  establishing  more  cost-to-cost 
and  cost-to-non-cost  CERs. 

CERs  are  a  simpler  parametric  technique  to  understand.  The  contractors  who  have 
made  the  sizeable  investment  to  take  this  retrospective  look  at  their  own  cost  history 
admit  how  difficult  and  expensive  it  is  to  get  the  required  data  and  find  a  way  to  correlate 
the  cost  data  to  the  activity  accomplished  and  the  cost  drivers  and  products  that 
influenced  the  events.  Although  costly,  they  expect  the  activity  to  be  a  good  investment. 
Some  other  contractors  have  yet  to  commit  to  this  course  and  fail  to  see  the  value.  A 
Boeing  executive  stated  this  CER  development  activity  was  a  good  place  to  start.  Start 
small  and  at  least  know  what  you've  done  and  establish  a  baseline  for  it  (Parametric  Cost 
Estimating  Joint  Industry/Govemment  Initiative  Workshop,  1998). 

The  government  has  never  discouraged  the  use  of  CERs.  The  CERs  were  not  used 
because  they  were  not  available  -  especially  cost-to-non-cost  CER’s.  The  use  of  CERs 
was  not  really  thought  of  as  a  parametric  technique,  yet  clearly  it  is.  When  based  on 
auditable  historical  data,  these  parametric  applications  do  give  it  all  -  quicker,  faster, 
better,  provided  the  inputs  of  those  CERs  can  be  accurately  estimated.  If  the  CER  is 
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hours  per  drawing;  if  historical  data  has  been  properly  used  to  develop  the  CER;  and  if 
the  number  of  drawings  can  be  accurately  estimated  for  the  current  project,  there  is 
probably  no  better  way  to  provide  an  estimate  for  drawing  labor  hours.  Accuracy  is  a 
function  of  the  accuracy  of  the  CER  and,  equally  important,  the  estimated  input  to  the 
CER.  CERs  remove  some  of  the  judgement  from  estimating.  The  government  does  not 
prefer  the  use  of  purely  judgement  estimates.  Judgement  estimates  usually  cause 
negotiation  problems.  One  LMTAS  executive  stated  they  are  formalizing  what  they  have 
always  done  or  what  they  should  have  been  doing  (Parametric  Cost  Estimating  Joint 
Industry/Govemment  Initiative  Workshop,  1998). 

Because  of  CERs,  it  is  easy  to  understand  why  the  PCEI  can  claim  achievements  of 
quicker,  faster,  and  more  accurate.  The  expanded  use  of  CERs  would  formalize  and 
encourage  quantitative  traceable  estimates.  Once  the  CERs  are  properly  developed,  all 
that  is  needed  is  the  task  and  the  magnitude  of  the  task  to  develop  the  input  to  the  CER. 
During  the  conceptualization  and  development  phases,  these  tasks  and  magnitudes  are  not 
known  or  cannot  be  quantified.  Cost-to-non-cost  CERs  become  more  essential  because 
the  goal  is  to  relate  cost  directly  to  the  product. 

The  more  complex  parametric  applications  involve  multiple  CER  parametric 
models.  The  inputs  are  no  longer  how  many  drawings  or  pages  of  output  are  expected 
because  that  information  is  not  available.  The  inputs  are  more  general  and  product 
oriented.  The  inputs  can  be  as  general  as  the  expected  weight,  size,  material,  throughput, 
packaging,  speed,  design  complexity,  and  many  other  higher  and  lower  level  product 
characteristics  or  performance  specifications.  The  difference  between  the  use  of  these 
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complex  models  and  simple  tasks  and  product  oriented  CERs  is  very  significant.  An 
unfamiliar  user  of  the  complex  model  will  have  no  ability  to  relate  the  estimate  to  his/her 
common  sense  as  the  correlation  between  the  estimate  and  the  task  gets  lost  in  the  black 
box  nature  of  the  parametric  model.  While  that  is  a  drawback,  there  is  a  significant 
upside  -  potential  gains  in  accuracy  and  speed  in  certain  situations. 

During  the  Parametric  Cost  Estimating  Joint  Industry/Govemment  Initiative,  a  great 
example  of  using  a  parametric  model  to  estimate  conceptual  and  preliminary  design  cost 
for  advanced  trans-atmospheric  and  orbital  vehicles  (Smith,  1998:2).  The  model  is  based 
on  nearly  thirty  years  of  system  level  data  provided  to  the  Redstar  database  of  the  DOD 
Cost  Analysis  and  Improvement  Group.  The  model  was  tested  by  establishing  concurrent 
estimates  and  making  comparisons  to  the  established  PRICE  and  SEER  commercial 
models.  For  unknown  reasons,  parametric  modeling  was  not  used  when  proceeding  to 
the  hardware  development  phase. 

General  Electric  (GE)  Aircraft  Engines  claims  significant  productivity  gains  from 
the  development  of  their  Cost  Offering  Method  for  Affordable  Propulsion  Engineering 
Acquisition  and  Test  (COMPEAT)  model  on  the  Joint  Strike  Fighter  program.  The 
model  was  created  by  engineering  and  military  proposal  groups  to  make  faster,  more 
accurate,  and  less  expensive  cost  estimates.  COMPEAT$  has  multiple  uses  including 
military  engine  development  proposals,  commercial  engine  studies,  conceptual/ 
preliminary/detailed  design  studies,  competitive  analysis,  should  cost  analysis,  and  target 
costing  (Griffin  and  Steams,  1997:9;  Parametric  Cost  Estimating  Joint  Industry/ 
Government  Initiative  Workshop,  1998).  Raytheon  E-Systems  provides  another  example 
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of  contractor  developed  parametric  models  with  their  Production  Cost  Estimating  Model 
&  Bidding  Tool.  The  products  modeled  are  all  near  or  advancing  state  of  the  art 
communications  hardware. 

There  are  other  examples  of  parametric  model  development,  but  not  nearly  as  many 
as  the  PCEI  accomplishments  in  the  area  of  simple  CERs.  Two  main  reasons  exist  for 
this.  First,  in  both  the  NASA  and  GE  situations,  the  use  of  these  in-house  models  has 
limited  applications.  The  models  are  intended  to  be  used  for  estimating  the  cost  of 
something  new,  complex  and  different.  GE  stated  they  did  not  intend  to  use  COMPEAT 
very  much  in  firm  priced  proposals  (Griffin  and  Steams,  1997).  The  NASA  model  was 
only  used  for  conceptual  and  preliminary  design  studies. 

The  second  reason  for  the  fewer  accomplishments  on  parametric  models  is  time, 
cost,  commitment,  and  payback  concerns.  These  models  involve  most  of  the  same 
activities  required  to  develop  the  simple  CERs,  plus  the  additional  activities  to  integrate 
those  CERs  into  a  comprehensive  parametric  model.  There  is  a  common  thread  between 
NASA  spacecraft,  new  GE  jet  engines,  and  Raytheon  E-System  products.  The 
connection  is  something  new,  very  different,  and  unique;  something  for  which  there  is  not 
experience  building;  something  for  which  there  is  not  direct  cost  history.  Each  of  the 
three  companies/agencies  mentioned  also  regularly  develops  new  and  different  systems  , 
at  least  often  enough  to  justify  the  development  of  these  parametric  models.  Without  the 
PCEI,  it  is  not  likely  these  applications  would  exist  or  be  as  far  along. 

Even  with  the  PCEI,  the  use  of  the  models  is  regulated  by  the  creators  or  the  real 
stakeholders.  GE  is  unlikely  to  use  COMPEAT  when  other  more  quantifiable  techniques 
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are  available.  NASA  contractors  are  unlikely  to  use  NAFCOM  when  they  clearly  know 
their  task.  Why  then  is  LMTAS  using  PRICE  H  to  estimate  F-16  after  building  over 
3,700  aircraft?  Why  is  Northrop  Grumman  testing  the  use  of  PRICE  H  to  estimate  radar 
system  costs  in  their  third  production  lot?  A  possible  answer  was  obtained  in  a  series  of 
interviews  from  contractor  and  government  personnel  involved  in  one  of  these  efforts. 

In  procurement  involving  aircraft  follow-on  production,  a  DOD  contractor  used  a 
commercial  model.  The  commercial  model  was  not  validated  and  calibrated  in  the 
traditional  way  using  actual  cost  history.  Instead,  the  contractor  used  recently  negotiated 
cost  for  aircraft  of  a  similar  configuration  and  force  fit  the  model  to  those  negotiated 
values.  This  may  have  been  due  to  the  difficulty  of  obtaining  and  organizing  actual  cost 
history.  This  was  an  issue  since  parametrics  estimating  still  require  current  cost  or 
pricing  data.  Negotiated  cost  from  previous  efforts  are  not  the  most  current  cost  or 
pricing  data. 

Management  also  imposed  a  goal  to  definitze  the  effort  within  three  months  of 
receipt  of  the  contractor's  proposal.  Eight  months  later,  the  contractor  and  government 
were  still  several  percent  apart  at  the  cost  line.  The  effort  started  with  a  near  20  percent 
variance  between  government  and  contractor  positions.  On  a  mature  program,  where 
follow  on  buys  have  small  configuration  differences  and  those  differences  involve  large 
dollar  values,  the  estimating  approach  should  provide  a  better  baseline  than  a  20  percent 
variance.  Previous  use  of  these  models  on  development  and  conceptual  efforts  seemed 
appropriate.  The  ±25  percent  accuracy  was  understood  and  generally  accepted. 
However,  this  kind  of  accuracy  on  mature  programs  causes  some  concern.  The  models 
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must  achieve  at  least  as  high  a  level  of  precision  as  traditional  expectations.  Both  parties 
agree  that  calibration  is  the  key  issue  to  accuracy.  Calibration  quality  is  driven  by  the 
accuracy  of  the  cost  and  technical  data  used.  Once  the  data  is  audited  for  accuracy  and 
the  calibration  techniques  are  approved,  customer  and  contractor  can  be  satisfied  that  the 
estimating  tool  is  accurately  baselined.  Some  of  the  government  personnel  feel  this 
approach  has  promise,  others  are  skeptical.  One  government  analyst  concurred  that 
confidence  in  the  accuracy  of  the  model  comes  from  how  it  was  calibrated  and  stated  it 
was  not  that  difficult  to  use.  There  is  some  support  for  using  parametric  models  for 
follow-on  production  efforts.  It  is  believed  that  accuracy  concerns  can  be  properly 
addressed  once  an  agreement  on  data  used  to  calibrate  is  determined.  Other  government 
people  feel  this  is  not  the  right  approach  for  follow-on  production. 

Ultimately,  the  reasons  for  using  the  parametric  model  were  reduced  proposal 
preparation  costs  and  cycle  timesavings.  The  contractor  claims  the  parametric  model 
approach  will  reduce  the  span  time  by  approximately  50  percent  while  cutting  proposal 
preparation  costs  by  approximately  60  percent.  Cycle  time  on  a  proposal  includes  time  to 
negotiate  issues.  A  common  issue  concerns  today's  push  for  higher  productivity  and 
improved  quality  assurance.  The  customer  wants  its  share  of  those  savings.  The  models 
have  ways  of  accommodating  these  issues  but  do  not  make  contractor  and  government 
agreement  on  quantifying  the  impacts  of  current  and  future  changes  any  easier.  These 
issues  must  still  be  negotiated  which  does  take  time.  Eventually,  the  parties  could  end  up 
using  more  traditional  approaches  to  negotiate  the  effort. 
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There  are  local  efforts  underway  to  address  the  conflict  between  using  parametric 
models  on  follow-on  production  and  FAR  15  requirements  for  subcontract  data.  On  this 
effort,  the  contractor  provided  required  cost  or  pricing  on  subcontracts.  The  contractor 
believes  that  estimating  subcontractor  cost  with  historically  indexed  parametric  models  is 
a  low  risk  for  the  government  and  the  contractor  as  long  as  the  product  configuration 
remains  the  same  or  similar.  However,  since  contractors  are  required  by  law  to  get 
current  quotes  and  evaluate  whether  they  are  fair  and  reasonable,  this  can  negate  the  cycle 
time  reductions  achieved  by  using  the  parametric  process.  Some  possible  solutions  under 
consideration  include:  1)  periodic  data  and  calibration  updates  similar  to  forward  pricing 
rate  agreements  and  designed  to  meet  TINA  requirements,  2)  TINA  waivers  on 
subcontractor  cost  or  pricing  data,  3)  regulatory  change  -  parametric  or  other  analysis  to 
determine  reasonable  price  from  subcontractor,  4)  definition  change  of  cost  or  pricing 
data  for  parametric  models,  5)  TINA  exemption  for  parametrics  ,  and  6)  leave  contractor 
at  risk  of  TINA  non-compliance. 

The  encouragement  of  the  use  of  CERs  by  the  PCEI  should  have  a  positive  effect  on 
the  proposal  preparation  and  negotiation  process,.  The  initiative  should  be  commended 
for  that  effort.  Conversely,  the  PCEI  may  be  encouraging  the  use  of  complex  parametric 
applications  in  situations  where  its  use  is  suspect. 

Recommendations.  Be  more  specific  about  the  type  of  parametric  application  under 
discussion. 
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Recommendations  for  Further  Research 


Future  research  in  this  area  should  focus  on  quantifying  the  impediments  to  the 
expanded  use  of  parametrics.  There  are  several  areas  where  this  study  provides  a  basis  to 
proceed: 

1.  Determine  the  main  problems  in  gathering  and  organizing  data  and  developing 
parametric  models.  Develop  feasible  solutions  with  the  estimated  cost  of 
implementation. 

2.  Determine  why  Non-PCEI  Government  have  such  a  low  perception  of  the 
accuracy  of  parametric  models.  What  should  the  PCEI  attempt  to  do  to  increase  the 
confidence  of  acquisition  groups  outside  the  initiative  concerning  the  accuracy  of 
parametric  techniques? 

3.  Explore  possible  impacts  of  not  using  cost  or  pricing  data,  especially  on 
subcontracts. 

Ending  Comments 

The  major  findings  of  this  study  are  summarized  as  follows.  Except  on  the  issue  of 
CERs,  Non-PCEI  Government  were  consistently  lower  in  their  perception  of  parametrics. 
The  use  of  the  single  term  parametric  estimating  as  an  encompassing  description  for 
CERs  and  parametric  models  is  improper  because  there  is  a  difference  in  how  CERs  and 
parametric  models  are  perceived.  Several  barriers  to  implementation  still  exist. 

Examples  include  data,  model  development,  training,  accuracy  perceptions,  and 
acquisition  rules  including  subcontract  pricing. 
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How  extensively  parametric  estimating  is  ultimately  adopted  will  be  determined  by 
whether  its  use  makes  sense  to  the  office  and  people  responsible  for  the  procurement. 
Contracting  officers,  contract  negotiators,  and  pricing  specialist  will  need  concrete 
reasons  to  adopt  vastly  different  estimating  methods  over  those  that  they  are  familiar  with 
and  make  sense  to  them.  Since  the  authority  to  determine  a  price  is  fair  and  reasonable  is 
delegated  to  these  lower  levels,  the  use  of  a  parametric  estimating  approach  must  appeal 
to  the  working  level  users.  It  must  provide  a  confidence  at  least  equal  to  conventional 
methods  currently  in  use.  If  expanded  parametrics  use  cannot  do  this,  its  future  may  be 
bleak  beyond  the  current  applications. 

A  summary  of  the  Deputy  SAF/AQ’s  comments  at  the  March  conference  provide  a 
fitting  end  to  this  thesis.  General  Anderson  wants  this  done  faster.  Data  capture, 
analysis,  and  maintenance  is  the  key.  Define  terms  clearly  so  we  all  clearly  understand 
what  we  are  saying.  Traning  for  both  sides,  government  and  contractor,  are  also  essential 
(Raney,  1998). 
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1 .  The  use  of  parametric  estimating  techniques  has  increased  in  the  past  3  years.  1  2  3  4  5 

2.  The  government  and  industry  have  been  successful  in  expanding  the  use  of 


parametric  estimation.  1  2  3  4  5 

3.  More  effort  is  needed  by  industry  to  expand  the  use  of  parametric  estimating.  1  2  3  4  5 

4.  More  effort  is  needed  by  the  Government  to  expand  the  use  parametric  estimating.  1  2  3  4  5 

5.  The  use  of  parametric  estimating  techniques  should  be  expanded  in  DoD  contracts.  1  2  3  4  5  ’ 

6.  Regulations  have  prevented  full  implementation  of  parametric  estimating  techniques.  1  2  3  4  5 

7.  Sufficient  funds  and  time  have  been  invested  by  industry  to  expand  the  use  of 

parametric  estimating.  1  2  3  4  5 

8.  Sufficient  funds  and  time  have  been  invested  by  the  Government  to  expand  the  use  of 

parametric  estimating.  1  2  3  4  5 

9.  The  need  for  accuracy  limits  expansion  of  parametric  techniques.  1  2  3  4  5 

10.  The  availability  of  data  limits  the  expanded  use  of  parametric  techniques.  1  2  3  4  5 

1 1 .  Customer  acceptance  limits  the  expanded  use  of  parametric  techniques.  1  2  3  4  5 

SECTION  2  UTILITY  (ACCURACY,  ACQUISITION  PHASE,  CONTRACT  TYPE) 

For  this  section,  the  term  Paramefrjc_istimate  refers  to  In-house  or  Commercial  Models,  not  simple  CERS. 

1 .  Parametric  estimating  should  only  be  used  on  cost  type  contracts.  1  2  3  4  5 

2.  If  information  is  limited,  parametric  estimating  techniques  should  be  used.  1  2  3  4  5 

3.  Parametric  estimating  is  more  appropriately  used  during  the  concept/development 

phase  of  a  program.  1  2  3  4  5 

4.  Parametric  estimating  techniques  can  be  used  to  estimate  change  orders.  1  2  3  4  5 

5.  The  amount  of  cost  information  available  determines  what  method  should  be  used.  1  2  3  4  5 

6.  If  sufficient  cost  information  is  available,  parametrics  should  not  be  used.  1  2  3  4  5 

7.  Contract  type  is  a  consideration  for  when  parametrics  should  be  used.  1  2  3  4  5 

8.  Parametrics  should  only  be  used  to  prepare  initial  budgetary  or  rough  estimates.  1  2  3  4  5 

9.  The  accuracy  of  establishing  contract  price  is  more  important  than  the  efficiency  of 

establishing  contract  price.  1  2  3  4  5 

10.  Parametric  estimates  are  as  accurate  as  traditional  cost  build  up  approaches  on 

concept/budgetary  estimates.  1  2  3  4  5 

1 1 .  Parametrics  estimates  are  a  accurate  as  traditional  cost  build  up  approaches  on 

mature  program  estimates.  1  2  3  4  5 

12.  Parametric  estimates  are  as  accurate  as  traditional  cost  build  up  approaches  in 

forecasting  productions  costs.  1  2  3  4  5 

13.  Parametric  estimating  is  inappropriate  for  use  on  follow-on  production.  1  2  3  4  5 

14.  Risk  analysis  is  important  in  estimating.  1  2  3  4  5 

15.  Risk  analysis  is  more  important  in  parametric  estimates  than  conventional  estimates.  1  2  3  4  5 

Please  turn  the  page  over  to  continue 
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SECTION  3  PARAMETRIC  APPLICATIONS 

The  statements  below  refer  to  six  categories  of  parametric  applications.  Read  each  statement  and  FILL  IN  the  number  below 
that  best  indicates  how  strongly  you  agree  or  disagree  with  it.  Note:  Commercial  and  In-house  models  are  composed  of 
multiple  Cost-to-Cost  and  Non-Cost  Cost  Estimating  Relationships  (CERs). 

Neither 

Strongly  Disagree  Disagree  Agree  nor  Disagree  Agree  Strongly  Agree 

12  3  _ 4 _ 5 _ 


CERs  that  are  Cost  to  Cost 

(Example:  QA  is  21%  of  production  hours): 

1 .  _ Are  easy  to  develop. 

2.  _ Are  easy  to  use. 

3.  _ Are  frequently  used. 

4.  _ Provide  benefit  in  efficiency. 

5.  _ Provide  benefit  in  accuracy. 

6.  _ Should  be  used  at  any  phase  of  the  acquisition  cycle. 

Commercial  Models  that  estimate  the  entire  cost  of 
a  proposal  (Example:  PRICE  H) 

1 .  _ Are  easy  to  LEARN. 

2.  _ Are  easy  to  use. 

3.  _ Are  frequently  used. 

4.  _ Provide  benefit  in  efficiency. 

5.  _ Provide  benefit  in  accuracy. 

6.  _ Should  be  used  at  any  phase  of  the  acquisition  cycle. 

In-house  Models  that  estimate  only  significant  portions 
of  a  proposal: 

1 .  _ Are  easy  to  develop. 

2.  _ Are  easy  to  use. 

3.  _ Are  frequently  used. 

4.  _ Provide  benefit  in  efficiency. 

5.  _ Provide  benefit  in  accuracy. 

6.  _ Should  be  used  at  any  phase  of  the  acquisition  cycle. 


CERs  that  are  Cost  to  Non-Cost 
(Example:  Hours  per  page): 

1 .  _ Are  easy  to  develop. 

2.  _ Are  easy  to  use. 

3.  _ Are  frequently  used. 

4.  _ Provide  benefit  efficiency 

5.  _ Provide  benefit  in  accuracy. 

6.  _ Should  be  used  at  any  phase  of  the  acquisition  cycle. 

Commercial  Models  that  estimate  only  significant  portions 
of  a  proposal  (Example:  COCOMO,  Software  Hours): 

1 .  _ Are  easy  to  LEARN. 

2.  _ Are  easy  to  use. 

3.  _ Are  frequently  used. 

4.  _ Provide  benefit  efficiency 

5.  _ Provide  benefit  in  accuracy. 

6.  _ Should  be  used  at  any  phase  of  the  acquisition  cycle. 

In-house  Models  that  estimate  the  entire  cost  of  a 
proposal: 

1 .  _ Are  easy  to  develop. 

2.  _ Are  easy  to  use. 

3.  _ Are  frequently  used. 

4.  _ Provide  benefit  efficiency 

5.  _ Provide  benefit  in  accuracy. 

6.  _ Should  be  used  at  any  phase  of  the  acquisition  cycle. 


SECTION  4 

The  statements  below  refer  to  the  main  challenges  or  barriers  to  parametric  estimating.  Identify  the  TOP  5  with  1  being  the 
BIGGEST  challenge  or  barrier,  2  the  next  biggest,  etc. _ _ _ 


database 

training 

accuracy 

risk  assessment 

TINA 

current  business  environment 
other  (piease  identify) 


Jack  of  Government  support 
Jack  of  Industry  support 
utility/usefulness 
model  development 
difficulty  of  use 

_acquisition  rules  and  regulations  (please  identify) 


Section  5  Demographic  Information 

This  section  of  the  survey  contains  several  items  dealing  with  personal  characteristics.  This  information  will  be  used  to  obtain 
a  picture  about  experience  and  differences  in  government  versus  contractor  perspectives. 

1 .  How  much  experience  do  you  have:  _ years 

2.  What  best  describes  the  work  that  you  do  (i.e.  price  analyst,  estimator,  contracts  etc)? _ 

3.  What  organization  do  you  work  for  (i.e.  industry,  consultant,  DCAA,  DCMC,  Govt)?  _ _ _ 

4.  How  many  estimates  have  you  prepared  or  analyzed  using  parametric  models? _ 

5.  How  many  proposals  (contract  prices)  have  you  prepared  or  analyzed  using  parametric  models? _ 

Thank  you  for  taking  the  time  to  complete  this  questionnaire. 

Any  comments  you  would  like  to  make  about  the  questionnaire  or  implementing  parametrics  are  welcome. 


91 


Bibliography 


Aeronautical  Systems  Division  (ASD).  The  Pricer’s  Estimating  Handbook.  Fairborn 
OH:  Wallace  &  Co.,  1989. 

“Allocation  of  Resources  and  Internal  Guidance,”  Parametric  Cost  Estimating  Initiative 
Newsletter.  December  1996:  8. 

Anderson,  Frank  J.,  Jr.  Director,  Contracting  Director  at  Aeronatical  Systems  Center, 
WPAFB  OH.  Personal  Correspondence.  1  May  97. 

Apgar,  H.  E.  and  J.  M.  Daschbach.  “Analysis  of  Design  Through  Parametric  Cost 
Estimation  Techniques,”  International  Society  of  Parametric  Analysis.  5-  13  (1987). 

Armed  Services  Procurement  Manual  for  Contract  Pricing.  ASPM  No  1 .  Commercial 
Clearing  House.  Chicago  IL,  1975. 

Biery,  Fred,  D.  Hudack,  and  G.  Shishu.  “Improving  Risk  Analysis,”  Journal  of  Cost 
Analysis.  57-85  (Spring  1994). 

Black,  Robert.  “A  Bibliography  of  Subjective  Elicitation  with  Emphasis  on  Risk 
Identification.”  Journal  of  Cost  Analysis:  61-80  (Spring  1997). 

Black,  Bob.  “Estimating  of  Input  Uncertainty  in  Parametric  Estimating,”  Boeing  Cost 
Library  No  1297N.  Grumman  Aerospace  Corporation,  Bethpage  NY,  1987. 

Brandstetter,  Ronald.  “In-house  Models,”  Parametric  Cost  Estimating  Initiative 
Newsletter.  August  1996:  6-7. 

“Calibration/Validation,”  Parametric  Cost  Estimating  Initiative  Newsletter.  December 
1996:  11  -  13. 

Chopey,  Nicholas  P.  “How  Accurate  is  your  Cost  Estimate,”  Chemical  Engineering.  104: 
106  - 1 1 1  (July  1994). 

Churchill,  Gilbert  A.,  Jr.  “A  Paradigm  for  Developing  Better  Measures  of  Marketing 
Constructs,”  Journal  of  Marketing  Research.  XVI:  64  -  73  (February  1979). 

Collins,  Jim.  “Metrics  for  Success,”  Parametric  Cost  Estimating  Initiative  Newsletter. 
August  1996:  8  -  9. 


92 


Collins,  Jim  and  David  Eck.  "Metrics  and  Questions  and  Answers  on  Current  Issues," 
Briefing  prepared  by  the  Co-Chairmen  of  the  Parametrics  Initiative  Steering 
Committee,  4th  Quarter  1997a. 

— .  “Working  Group  Messages,”  Parametric  Cost  Estimating  Initiative  Newsletter, 
December  1997b:  1-3 

Cooper,  Donald  R.  and  C.  William  Emory.  Business  Research  Methods.  New  York: 
The  McGraw-Hill  Company,  Inc.,  1995. 

Dane,  Francis  C.  Research  Methods.  Pacific  Grove  CA:  Brooks/Cole  Publishing 
Company,  1990. 

Dean,  E.  B.  “Parametric  Cost  Deployment,”  Excerpt  from  unpublished  article,  n.pag. 
WWWeb.  http://miiuno.lare.nasa.gov/dfc/societis/Ispa.html.  15  October  1997. 

De  la  Garza,  Jesus  and  Khalil  C.  Rouohana.  “Neural  Networks  versus  Parameter-Based 
Applications  in  Cost  Estimating,”  Cost  Engineering.  37:  14-18  (February  1995). 

Department  of  Defense.  Parametric  Cost  Estimating  Handbook.  Joint  Government/ 
Industry  Initiative.  Arlington  VA:  NAVSEA,  Fall  1995. 

Doraiswamy,  Shri  and  Maurice  J.  Hunt.  “Training  Estimating,”  AACE  Transactions: 
Q.M.4.1  -  Q.M.4.4  (1994). 

Druyun,  Darleen  A.,  Principal  Deputy  Assistant  Secretary  (Acquisition  and 
Management),  Office  of  the  Assistant  Secretary,  Washington  DC.  Personal 
Correspondence.  30  January  1997. 

“Executive  Summary  on  the  Results  of  the  PCEI ESC/WG  Site  Visit,”  Parametric  Cost 
Estimating  Initiative  Newsletter.  April  1997a:  11-12. 

“Executive  Summary  on  the  Results  of  the  PCEI  ESC/WG  Site  Visit,”  Parametric  Cost 
Estimating  Initiative  Newsletter.  December  1997b:  11-12. 

Fowler,  Joe.  Director  of  Estimates,  Lockheed  Martin  Tactical  Aircraft  Systems,  Fort 
Worth  TX.  Office  Meetings.  March  -  April  1997. 

Gall,  Meredith  D.,  Walter  R.  Borg,  and  Joyce  P.  Gall.  Education  Research:  An 
Introduction.  White  Plains  NY:  Logman  Publishers  USA,  1996. 


93 


Gardner,  Thomas  E.  and  Stephan  M.  Passarello.  A  Parametric  Estimating  Model  for 
Flight  Simulator  Acquisition.  MS  thesis,  AFIT/LSSR/41-81.  School  of  Systems  and 
Logistics,  Air  Force  Institute  of  Technology  (AU),  Wright-Patterson  AFB  OH,  June 
1981  (A103237). 

Geaney,  Columba  Michael.  “A  Strategy  for  the  Development  of  a  Cost  Engineering 
Database,”  Cost  Engineering.  39:  30  -  35  (April  1997). 

Graziano,  Anthony  M.  and  Michael  L.  Raulin.  Research  Methods:  A  Process  of  Inquiry. 
New  York:  Harper  Collins  Publishers,  Inc.,  (1989). 

Griffin,  Carolyn  and  Marshall  Steams.  Team  Leaders  at  General  Electric  Aircraft 
Engines,  Cincinnati  OH.  Office  meetings,  Spring  1997a. 

— .  “Executive  Summary  on  the  Results  of  the  PCEI/  WG  Site  Visit,”  Parametric  Cost 
Estimating  Initiative  Newsletter.  December  1997b:  9. 

Hancock,  Dain.  “Quotes,”  Parametric  Cost  Estimating  Initiative  Newsletter.  December 
1996:2. 

Hargrove,  Noel,  Director  of  Pricing.  “Parametric  Cost  Estimating:  A  Contractor  View,” 
Boeing  Cost  Library  No  1491N.  Allied  Company  Report,  Bunker  Ramo  Electronic 
Systems,  1985. 

Herding,  Vergil,  Air  Force  Material  Command.  “An  Overview  of  the  Parametric  Cost 
Estimating  Initiative.”  Address  to  the  Parametric  Cost  Estimating  Joint  Industry/ 
Government  Initiative  Workshop,  Los  Angeles  CA,  3-4  March  1998. 

Labuschagne,  Adri.  “Qualitative  Research  -  Air  Fairy  or  Fundamental?”  n.pag. 

WWWeb,  http://www.mrc.ac.za/mrcnews/june96/qualit.htm.  16  April  1998. 

Larsen,  Harry  T.  “On  the  Value  of  Cost  Estimating  Accuracy  Under  Conditions  of 
Competitive  Bidding,”  Cost  Engineering,  36:  25  -  27  (January  1994). 

Lederer,  Albert  L.  “Nine  Management  Guidelines  for  Better  Cost  Estimating,” 
Communications  of  the  ACM.  35:  50  -  59  (February  1992a). 

— .  “Putting  Estimates  on  Track,”  Computer  World.  26:  85  -  87  (August  1992b). 

Lewis,  Edwin  M.  and  Eugene  D.  Pearson.  The  AF  Cost  Estimating  Process:  Agencies 
Involved  and  Estimating  Techniques  Used.  MS  thesis,  AFIT/LSSR/5-77A.  School 
of  Systems  and  Logistics,  Air  Force  Institute  of  Technology  (AU),  Wright-Patterson 
AFB  OH,  June  1977  (A044101). 


94 


Lewis,  Joe.  Address  to  the  Parametric  Cost  Estimating  Joint  Industry/  Government 
Initiative  Workshop,  Los  Angeles  CA,  3-4  March  1998. 

Lubel,  Paul.  “Executive  Summary  on  the  Results  of  the  PCEI/  WG  Site  Visit,” 
Parametric  Cost  Estimating  Initiative  Newsletter.  December  1997:  7-8. 

Mann,  Tom  E.,  Director  of  Pricing,  Aeronautical  Systems  Center,  WPAFB  OH.  Personal 
interview.  1  May  1997. 

Marshall,  Catherine  and  Gretchen  B.  Rossman.  Designing  Qualitative  Research. 
Newbury  Park  CA,  Sage  Publications,  Inc,  1991. 

Mathis,  William,  Senior  Consultant.  “Cost  Schedule  Control  Systems  Criteria  (C/SCSC): 
Boon  to  the  Parametric  Analyst,”  Boeing  Cost  Library  No.  1723N.  Decision 
Planning  Corp,  Costa  Mesa  CA,  undated. 

McDonald,  Donald  F.,  Jr.  “Cost  Estimates  in  Litigation  Disputes,”  AACE  Transactions: 
D.3.1  -  D.3.7  (1992). 

Ogunlana,  Olu.  “Learning  from  Experience  in  Design  Cost  Estimating,”  Construction 
Management  &  Economics.  9: 133  -  150  (April  1990). 

Parametric  Cost  Estimating  Course  Manual.  Federal  Publications  Inc,  1987. 

Parametric  Cost  Estimating  Joint  Industry/  Government  Initiative  Workshop,  Los 
Angeles  CA,  3-4  March  1998. 

Philpott,  M.  L.,  C.  S.  Warrington,  E.  A.  Branstad,  R.  David,  and  R.  P.  Nita.  “A 

Parametric  Contract  Modeler  for  DFM  Analysis,”  Journal  of  Manufacturing  Systems, 
15:  256-267  (1996). 

Rany,  Terry.  Acting  Associate  Deputy  Assistant  Secretary  (Contracting)  and  the 
Contract  Policy  Division  Chief,  Office  of  the  Deputy  Assistant  Secretary 
(Contracting),  Assistant  Secretary  of  the  Air  Force  (Acquisition).  "Encouraging 
Implementation  of  Parametric  Techniques  in  1998."  Address  to  the  Parametric  Cost 
Estimating  Joint  Industry/Govemment  Initiative  Workshop,  Los  Angeles  CA,  3-4 
March  1998. 

Remer,  Donald  S.  and  Harry  Buchanan.  “Estimation  Knowledge:  A  Conceptual 
Framework  for  Cost  Estimation,”  Cost  Engineering.  38:  7-1 1  (March  1993). 


95 


Rose,  Howard  C.  and  James  D.  High.  An  Extension  of  Cost  Estimating  Relationships  for 
Air  Frames  of  Remotely  Piloted  Vehicles.  MS  thesis,  AFIT/LSA/SM/74D-5.  School 
of  Systems  and  Logistics,  Air  Force  Institute  of  Technology  (AU),  Wright-Patterson 
AFB  OH,  December  1974  (A003352). 

Smith,  Alice  E.  and  Anthony  K.  Mason.  “Cost  Estimation  Predictive  Modeling: 

Regression  versus  Neural  Network,”  Engineering  Economist  42:  137  -  161  (1997). 

Smith,  Keith.  "Parametric  Applications  of  the  NASA/Air  Force  Cost  Model 
(NAFCOM96)."  Address  to  the  Parametric  Cost  Estimating  Joint 
Industry/Govemment  Initiative  Workshop,  Los  Angeles  CA,  3-4  March  1998. 

Spector,  Eleanor  R.,  Director,  Defense  Procurement,  Office  of  the  Under  Secretary  of 
Defense,  Washington  DC.  Personal  Corrspondence.  28  August  1995. 

Stanley,  Wayne  A.  “Parametric  -  Wave  of  the  Future,  Customer  Acceptance  -  A  Key  to 
Success,”  Paper  from  General  Dynamics  Pomona  Division,  1981 . 

"Statutory/Policy  Changes,”  AFMC  Guide  to  Acquisition  Reform,  1  -  8.  WWWeb, 
http://www.afinc.wpafb.af.mil/organ...qref/statpoli/statpol3 .htm#execsum.  28  May 
1998. 

Tesch,  Renata.  Qualitative  Research.  Bristol  PA:  The  Falmer  Press,  1990. 

Webster’s  Ninth  New  Collegiate  Dictionary,  Merriam-Webster  Inc.  Publishers,  1984. 

Widmann,  E.  R.,  I.  J.  Vincenzini,  and  M.  Emstoff.  “Development  of  a  Parametric  Cost 
Model  for  Systems  Engineering,”  Source  Unknown.  433  -  442,  (1997). 

“Working  Relationships,”  Parametric  Cost  Estimating  Initiative  Newsletter.  April  1997: 
16. 

Zhang,  Y.  F.,  J.  Y.  H.  Fuh,  and  W.  T.  Chan.  “Feature-based  Cost  Estimating  for 
Packaging  Products  Using  Neural  Networks,”  Computers  in  Industry.  32:  95  -  113 
(June  1996). 


* 


96 


Vita 


Ms.  Patricia  Blakely  was  bom  on  9  August  1958  in  Columbus,  Georgia.  She 
graduated  from  Wayne  High  School  in  Huber  Heights,  Ohio  in  1976  and  attended  Miami 
University  in  Oxford,  Ohio.  In  1980,  she  graduated  with  a  Bachelor  of  Science  in 
Business  Administration,  majoring  in  Finance.  She  began  her  contracting  career  as  a 
procurement  analyst  at  the  Defense  Electronic  Supply  Center  in  Kettering,  Ohio.  In 
1984,  she  transferred  to  the  Aeronautical  Systems  Center,  WPAFB,  Ohio  where  she 
worked  as  a  contract  negotiator  in  the  Reconnaissance  Strike/Electronic  Warfare  Systems 
Program  Office  (SPO),  Strategic  Systems  SPO,  Propulsion  SPO,  Simulator  SPO,  F-16 
SPO,  Specialized  Contracting  Office,  and  the  Resource  Management  Office.  In  1989,  she 
received  her  contracting  officer  warrant.  She  was  selected  to  attend  long-term  full-time 
training  at  the  Air  Force  Institute  of  Technology  and,  in  May  1997,  entered  the  Graduate 
School  of  Logistics  and  Acquisition  Management  in  Contract  Management.  Ms.  Blakely 
is  married  to  Kelly  Blakely  and  has  two  children:  a  daughter  Jessica  and  a  son  Casey. 

Permanent  Address:  3750  Southfield  Drive 

Beavercreek,  Ohio  45434 


X 


97 


Vita 


Mr.  Rudolph  Haynesworth  was  bom  on  3  March  1955  in  Cleveland,  Ohio.  He 
graduated  from  Shaw  High  School  in  East  Cleveland,  Ohio  in  1973  and  attended  the 
General  Motors  Institute  in  Flint,  Michigan.  In  1978,  he  graduated  with  a  Bachelor  of 
Science  in  Industrial  Management,  majoring  in  Production  Management.  He  enlisted  in 
the  Air  Force  in  April  1979,  and  was  commissioned  a  Second  Lieutenant  in  September 
1979.  He  served  six  years  active  duty  for  the  Air  Force  Contract  Management  Division, 
Air  Force  Systems  Command,  at  the  Northrop  Corporation,  Hawthorne,  California,  and 
North  American  Rockwell,  Columbus,  Ohio  as  an  in-plant  program  manager  and 
supervisory  engineer.  He  separated  from  the  Air  Force  in  September  1985  to  work  for 
Aeronautical  Systems  Division  (ASD/ASC),  WPAFB,  Ohio,  Pricing  Division.  In 
nineteen  years  with  the  Air  Force,  he  has  provided  service  on  the  B-l,  F-16,  F-15,  and 
Navy  F-18  programs  and  many  high  dollar  procurement  actions  on  subsystem  programs. 
He  has  also  been  the  ASC  focal  point  for  several  global  issues,  including  parametric 
estimating.  He  was  selected  to  attend  long-term  full-time  training  at  the  Air  Force 
Institute  of  Technology  and,  in  May  1997,  entered  the  Graduate  School  of  Logistics  and 
Acquisition  Management  in  Contract  Management.  Mr  Haynesworth  is  married  to  the 
former  Alma  Singleton  of  Buffalo,  New  York.  They  have  one  son,  Rudolph  III. 

Permanent  Address:  6268  Weybridge  Drive 
Trotwood,  Ohio  45426 


98 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  074-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  reponse.  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  the  collection  of  information  induding 
suggestions  for  redueting  this  burden  to  Washington  Headquarters  Services.  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202^1302, 
and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (070443188),  Washington,  DC  20503 - - - - - — - - — ■ — 

1  AGFNCY  USE  ONLY  (Leave  I  2.  REPORT  DATE  I  3.  REPORT  TYPE  AND  DATES  COVERED 


1.  AGENCY  USE  ONLY  (Leave 
blank) 


2.  REPORT  DATE 

September  1998 


Master’s  Thesis 


4.  TITLE  AND  SUBTITLE 

AN  EXPLORATORY  STUDY  OF  THE  USE  OF  PARAMETRIC  ESTIMATING 
IN  DEPARTMENT  OF  DEFENSE  CONTRACTING  _ 


6.  AUTHOR(S) 

Patricia  Blakely,  GS-12 
Rudolph  Haynesworth,  GS-13 


7.  PERFORMING  ORGANIZATION  NAMES(S)  AND  ADDRESS(S) 

Air  Force  Institute  of  Technology 

2950  P  Street 

WPAFB  OH  45433-7765 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

AFIT/GCM/LAS/98S-1 


9.  SPONSORING  I  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

SAF/AQCP  Attn:  EricKattner 
1060Air  Force  Pentagon 

Washington  DC  20330-1060 _ _ 


11.  SUPPLEMENTARY  NOTES 


10.  SPONSORING  /  MONITORING 
AGENCY  REPORT  NUMBER 


12a7DiSTRIBUTION  /  AVAILABILITY  STATEMENT  12b.  DISTRIBUTION  CODE 

Approved  for  public  release;  distribution  unlimited 
13.  ABSTRACT  (Maximum  200  Words) 

An  initiative  to  expand  the  use  of  parametric  estimating  in  government  contracting  has  enjoyed  only  marginal  success.  Some 
reasons  for  the  slower  pace  of  implementation  are  the  lack  of  availability  of  historical  data,  training,  acquisition  rules,  utility,  and 
accuracy  concerns.  Those  associated  with  the  Parametric  Cost  Estimating  Initiative  (PCEI)  have  a  higher  perception  of  parametric 
utility  and  accuracy  than  those  not  associated  with  the  initiative.  The  PCEI  advocates  using  parametric  techniques  in  any  situation, 
not  just  the  concept/development  phases  advocated  by  most  literature. 

Parametric  estimating  is  a  catch-all  term  for  several  different  types  of  statistically  based  methodologies,  ranging  from  simple 
CERs  to  complex  models.  Each  type  has  different  implementation  requirements.  Using  the  single  term  parametrics  can  be  very 
deceiving,  making  it  difficult  for  management  to  allocate  resources  necessary  for  implemetation. 

PCEI  and  non-PCEI  acquisition  groups  have  similar  perceptions  of  accuracy  and  utility  for  CERs,  but  different  perceptions  for 
more  complex  parametric  models.  The  PCEI  has  been  successful  expanding  the  use  of  parametrics,  though  not  significantly  enough 
to  be  persuasive.  Most  of  the  PCEI  successes  have  been  related  to  CERs,  not  complex  models.  Advances  in  complex  model  use  has 
occurred  with  contractors  who  routinely  develop  state  of  the  art  hardware. 


14.  Subject  Terms 

parametric  cost  estimating,  acquisition  reform,  cost  analysis,  estimating  techniques,  cost  estimating 
relationships 


15.  NUMBER  OF  PAGES 
110 


16.  PRICE  CODE 


17.  SECURITY  CLASSIFICATION  18.  SECURITY  CLASSIFICATION  19.  SECURITY  CLASSIFICATION  20.  LIMITATION  OF  ABSTRACT 
OF  REPORT  OF  THIS  PAGE  OF  ABSTRACT 

UNCLASSIFIED  I  UNCLASSIFIED  UNCLASSIFIED  | _ UL _ 

NSN  7540-01-280-5500  Standard  Form  298  (Rev.  2-89) 

Prescribed  by  ANSI  Std.  Z39-18 
298-102 


AFIT  Control  Number  AFIT/GCM/LAS/98S-1 


AFIT  RESEARCH  ASSESSMENT 

The  purpose  of  this  questionnaire  is  to  determine  the  potential  for  current  and  future  applications 
of  AFIT  thesis  research.  Please  return  completed  questionnaire  to:  AIR  FORCE  INSTITUTE 
OF  TECHNOLOGY/LAC,  2950  P  STREET,  WRIGHT-PATTERSON  AFB  OH  45433-7765. 
Your  response  is  important.  Thank  you. 

1.  Did  this  research  contribute  to  a  current  research  project?  a.  Yes  b.  No 


2.  Do  you  believe  this  research  topic  is  significant  enough  that  it  would  have  been  researched  (or 
contracted)  by  your  organization  or  another  agency  if  AFIT  had  not  researched  it? 

a.  Yes  b.  No 


3.  Please  estimate  what  this  research  would  have  cost  in  terms  of  manpower  and  dollars  if  it  had 
been  accomplished  under  contract  or  if  it  had  been  done  in-house. 

Man  Years _  $ _ 


4.  Whether  or  not  you  were  able  to  establish  an  equivalent  value  for  this  research  (in  Question 
3),  what  is  your  estimate  of  its  significance? 

a.  Highly  b.  Significant  c.  Slightly  d.  Of  No 

Significant  Significant  Significance 

5.  Comments  (Please  feel  free  to  use  a  separate  sheet  for  more  detailed  answers  and  include  it 
with  this  form): 


Name  and  Grade 


Organization 


Position  or  Title 


Address 


